May 06 17:10:28.442535 ip-10-0-130-186 systemd[1]: Starting Kubernetes Kubelet...
May 06 17:10:28.879331 ip-10-0-130-186 kubenswrapper[2560]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 06 17:10:28.879331 ip-10-0-130-186 kubenswrapper[2560]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 06 17:10:28.879331 ip-10-0-130-186 kubenswrapper[2560]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 06 17:10:28.879331 ip-10-0-130-186 kubenswrapper[2560]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 06 17:10:28.879331 ip-10-0-130-186 kubenswrapper[2560]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 06 17:10:28.880773 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.880682    2560 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 06 17:10:28.883589 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883569    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 06 17:10:28.883589 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883585    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 06 17:10:28.883589 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883590    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 06 17:10:28.883589 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883593    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883598    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883604    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883608    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883612    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883616    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883620    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883624    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883628    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883639    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883643    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883647    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883651    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883655    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883659    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883663    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883666    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883670    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883674    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883678    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 06 17:10:28.883809 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883681    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 06 17:10:28.884306 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883685    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 06 17:10:28.884306 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883689    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 06 17:10:28.884306 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883693    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 06 17:10:28.884306 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883697    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 06 17:10:28.884306 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883700    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 06 17:10:28.884306 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883705    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 06 17:10:28.884306 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883709    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 06 17:10:28.884306 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883713    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 06 17:10:28.884306 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883717    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 06 17:10:28.884306 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.883721    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884380    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884392    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884398    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884403    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884408    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884413    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884418    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884422    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884429    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884433    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884438    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884443    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884447    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884452    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884456    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884460    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884464    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884468    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 06 17:10:28.884705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884475    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884479    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884484    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884488    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884492    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884497    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884502    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884508    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884514    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884519    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884523    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884527    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884531    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884535    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884539    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884544    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884549    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884553    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884558    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 06 17:10:28.885573 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884562    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884566    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884571    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884575    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884580    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884584    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884588    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884592    2560 feature_gate.go:328] unrecognized feature gate: Example
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884599    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884603    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884608    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884612    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884616    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884620    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.884624    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885242    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885251    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885255    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885259    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885264    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 06 17:10:28.886400 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885267    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885273    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885277    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885282    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885286    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885291    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885297    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885303    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885310    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885316    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885321    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885326    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885331    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885335    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885340    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885344    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885349    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885354    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885358    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885363    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 06 17:10:28.887232 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885367    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885371    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885375    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885380    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885384    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885388    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885392    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885396    2560 feature_gate.go:328] unrecognized feature gate: Example
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885400    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885406    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885412    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885417    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885421    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885425    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885429    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885433    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885437    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885441    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885445    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885449    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 06 17:10:28.887733 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885455    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885459    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885464    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885468    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885472    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885476    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885481    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885484    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885489    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885494    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885498    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885502    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885506    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885510    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885514    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885518    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885522    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885527    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885532    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 06 17:10:28.888362 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885536    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885541    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885545    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885549    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885555    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885559    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885563    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885568    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885572    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885577    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885581    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885586    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885590    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885594    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885598    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885602    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885607    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885611    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885615    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885619    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 06 17:10:28.888920 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885623    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.885627    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885733    2560 flags.go:64] FLAG: --address="0.0.0.0"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885744    2560 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885754    2560 flags.go:64] FLAG: --anonymous-auth="true"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885761    2560 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885769    2560 flags.go:64] FLAG: --authentication-token-webhook="false"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885775    2560 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885782    2560 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885789    2560 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885794    2560 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885799    2560 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885805    2560 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885810    2560 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885815    2560 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885820    2560 flags.go:64] FLAG: --cgroup-root=""
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885825    2560 flags.go:64] FLAG: --cgroups-per-qos="true"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885830    2560 flags.go:64] FLAG: --client-ca-file=""
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885835    2560 flags.go:64] FLAG: --cloud-config=""
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885840    2560 flags.go:64] FLAG: --cloud-provider="external"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885845    2560 flags.go:64] FLAG: --cluster-dns="[]"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885851    2560 flags.go:64] FLAG: --cluster-domain=""
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885856    2560 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885861    2560 flags.go:64] FLAG: --config-dir=""
May 06 17:10:28.889578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885866    2560 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885872    2560 flags.go:64] FLAG: --container-log-max-files="5"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885879    2560 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885883    2560 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885888    2560 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885894    2560 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885898    2560 flags.go:64] FLAG: --contention-profiling="false"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885903    2560 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885908    2560 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885913    2560 flags.go:64] FLAG: --cpu-manager-policy="none"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885918    2560 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885925    2560 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885930    2560 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885935    2560 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885940    2560 flags.go:64] FLAG: --enable-load-reader="false"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885945    2560 flags.go:64] FLAG: --enable-server="true"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885949    2560 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885957    2560 flags.go:64] FLAG: --event-burst="100"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885961    2560 flags.go:64] FLAG: --event-qps="50"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885966    2560 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885971    2560 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885976    2560 flags.go:64] FLAG: --eviction-hard=""
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885983    2560 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885988    2560 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885993    2560 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 06 17:10:28.890188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.885998    2560 flags.go:64] FLAG: --eviction-soft=""
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886002    2560 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886010    2560 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886015    2560 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886020    2560 flags.go:64] FLAG: --experimental-mounter-path=""
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886026    2560 flags.go:64] FLAG: --fail-cgroupv1="false"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886030    2560 flags.go:64] FLAG: --fail-swap-on="true"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886036    2560 flags.go:64] FLAG: --feature-gates=""
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886042    2560 flags.go:64] FLAG: --file-check-frequency="20s"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886047    2560 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886052    2560 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886058    2560 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886063    2560 flags.go:64] FLAG: --healthz-port="10248"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886068    2560 flags.go:64] FLAG: --help="false"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886073    2560 flags.go:64] FLAG: --hostname-override="ip-10-0-130-186.ec2.internal"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886078    2560 flags.go:64] FLAG: --housekeeping-interval="10s"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886083    2560 flags.go:64] FLAG: --http-check-frequency="20s"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886087    2560 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886093    2560 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886099    2560 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886104    2560 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886109    2560 flags.go:64] FLAG: --image-service-endpoint=""
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886114    2560 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 06 17:10:28.890880 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886119    2560 flags.go:64] FLAG: --kube-api-burst="100"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886124    2560 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886129    2560 flags.go:64] FLAG: --kube-api-qps="50"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886133    2560 flags.go:64] FLAG: --kube-reserved=""
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886139    2560 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886143    2560 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886164    2560 flags.go:64] FLAG: --kubelet-cgroups=""
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886169    2560 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886174    2560 flags.go:64] FLAG: --lock-file=""
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886179    2560 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886184    2560 flags.go:64] FLAG: --log-flush-frequency="5s"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886191    2560 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886199    2560 flags.go:64] FLAG: --log-json-split-stream="false"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886204    2560 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886213    2560 flags.go:64] FLAG: --log-text-split-stream="false"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886217    2560 flags.go:64] FLAG: --logging-format="text"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886222    2560 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886227    2560 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886232    2560 flags.go:64] FLAG: --manifest-url=""
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886237    2560 flags.go:64] FLAG: --manifest-url-header=""
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886243    2560 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886248    2560 flags.go:64] FLAG: --max-open-files="1000000"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886255    2560 flags.go:64] FLAG: --max-pods="110"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886260    2560 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886265    2560 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 06 17:10:28.891457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886269    2560 flags.go:64] FLAG: --memory-manager-policy="None"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886274    2560 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886279    2560 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886283    2560 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886289    2560 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886301    2560 flags.go:64] FLAG: --node-status-max-images="50"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886306    2560 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886311    2560 flags.go:64] FLAG: --oom-score-adj="-999"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886316    2560 flags.go:64] FLAG: --pod-cidr=""
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886320    2560 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:3fc6c2cc09f271efd3cd2adb6c984c7cab48ea53dad824c952dee91afa8eaa20"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886330    2560 flags.go:64] FLAG: --pod-manifest-path=""
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886334    2560 flags.go:64] FLAG: --pod-max-pids="-1"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886339    2560 flags.go:64] FLAG: --pods-per-core="0"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886344    2560 flags.go:64] FLAG: --port="10250"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886349    2560 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886354    2560 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0e544e0d28e8cf97e"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886359    2560 flags.go:64] FLAG: --qos-reserved=""
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886364    2560 flags.go:64] FLAG: --read-only-port="10255"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886371    2560 flags.go:64] FLAG: --register-node="true"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886376    2560 flags.go:64] FLAG: --register-schedulable="true"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886381    2560 flags.go:64] FLAG: --register-with-taints=""
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886386    2560 flags.go:64] FLAG: --registry-burst="10"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886393    2560 flags.go:64] FLAG: --registry-qps="5"
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886397    2560 flags.go:64] FLAG: --reserved-cpus=""
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886402    2560 flags.go:64] FLAG: --reserved-memory=""
May 06 17:10:28.892115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886414    2560 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886419    2560 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886424    2560 flags.go:64] FLAG: --rotate-certificates="false"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886429    2560 flags.go:64] FLAG: --rotate-server-certificates="false"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886433    2560 flags.go:64] FLAG: --runonce="false"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886438    2560 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886443    2560 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886448    2560 flags.go:64] FLAG: --seccomp-default="false"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886453    2560 flags.go:64] FLAG: --serialize-image-pulls="true"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886458    2560 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886463    2560 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886467    2560 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886472    2560 flags.go:64] FLAG: --storage-driver-password="root"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886478    2560 flags.go:64] FLAG: --storage-driver-secure="false"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886483    2560 flags.go:64] FLAG: --storage-driver-table="stats"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886488    2560 flags.go:64] FLAG: --storage-driver-user="root"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886493    2560 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886497    2560 flags.go:64] FLAG: --sync-frequency="1m0s"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886502    2560 flags.go:64] FLAG: --system-cgroups=""
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886507    2560 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886516    2560 flags.go:64] FLAG: --system-reserved-cgroup=""
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886521    2560 flags.go:64] FLAG: --tls-cert-file=""
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886526    2560 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886532    2560 flags.go:64] FLAG: --tls-min-version=""
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886537    2560 flags.go:64] FLAG: --tls-private-key-file=""
May 06 17:10:28.892752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886544    2560 flags.go:64] FLAG: --topology-manager-policy="none"
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886549    2560 flags.go:64] FLAG: --topology-manager-policy-options=""
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886554    2560 flags.go:64] FLAG: --topology-manager-scope="container"
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886559    2560 flags.go:64] FLAG: --v="2"
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886567    2560 flags.go:64] FLAG: --version="false"
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886573    2560 flags.go:64] FLAG: --vmodule=""
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886580    2560 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.886585    2560 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886739    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886746    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886751    2560 feature_gate.go:328] unrecognized feature gate: Example
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886755    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886760    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886764    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886768    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886772    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886777    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886781    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886785    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886789    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886793    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 06 17:10:28.893427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886798    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886803    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886807    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886812    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886816    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886820    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886825    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886829    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886833    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886836    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886840    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886847    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886852    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886856    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886860    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886865    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886870    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886874    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886878    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886882    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 06 17:10:28.893934 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886886    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886891    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886895    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886899    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886905    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886911    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886915    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886919    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886923    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886928    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886933    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886937    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886942    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886946    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886951    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886955    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886959    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886963    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886967    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886971    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 06 17:10:28.894557 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886975    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886978    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886985    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886993    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.886997    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887001    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887006    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887012    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887016    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887020    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887025    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887029    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887033    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887037    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887040    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887044    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887048    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887052    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887057    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887061    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 06 17:10:28.895318 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887065    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887069    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887073    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887078    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887082    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887085    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887090    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887094    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887099    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887103    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887107    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887111    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.887114    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.888037    2560 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.895670    2560 server.go:530] "Kubelet version" kubeletVersion="v1.33.10"
May 06 17:10:28.895876 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.895686    2560 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895734    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895740    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895744    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895748    2560 feature_gate.go:328] unrecognized feature gate: Example
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895751    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895754    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895757    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895760    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895763    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895765    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895768    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895770    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895773    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895776    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895778    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895781    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895783    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895786    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895789    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 06 17:10:28.896307 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895791    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895794    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895796    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895799    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895802    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895805    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895808    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895810    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895813    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895815    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895818    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895820    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895823    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895826    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895829    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895831    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895834    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895836    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895839    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895841    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 06 17:10:28.896801 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895844    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895846    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895849    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895851    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895854    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895856    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895860    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895865    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895868    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895870    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895873    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895876    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895878    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895881    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895884    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895886    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895889    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895892    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895894    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 06 17:10:28.897350 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895897    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895900    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895902    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895905    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895908    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895910    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895913    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895916    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895918    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895922    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895924    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895927    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895930    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895933    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895935    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895938    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895940    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895944    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895947    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 06 17:10:28.897820 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895950    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895952    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895955    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895957    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895960    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895963    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895965    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895968    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.895970    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.895976    2560 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896069    2560 feature_gate.go:328] unrecognized feature gate: Example2
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896074    2560 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896078    2560 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896081    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896084    2560 feature_gate.go:328] unrecognized feature gate: NewOLM
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896087    2560 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 06 17:10:28.898309 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896090    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896092    2560 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896096    2560 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896099    2560 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896102    2560 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896105    2560 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896108    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896110    2560 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896113    2560 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896116    2560 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896118    2560 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896121    2560 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896123    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896125    2560 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896128    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896131    2560 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896134    2560 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896137    2560 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896139    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896142    2560 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 06 17:10:28.898705 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896145    2560 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896164    2560 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896169    2560 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896172    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896174    2560 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896177    2560 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896179    2560 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896182    2560 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896184    2560 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896187    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896190    2560 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896192    2560 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896196    2560 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896200    2560 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896204    2560 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896208    2560 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896211    2560 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896214    2560 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 06 17:10:28.899424 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896217    2560 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896220    2560 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896223    2560 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896225    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896228    2560 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896230    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896233    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896236    2560 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896238    2560 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896241    2560 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896243    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896246    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896249    2560 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896252    2560 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896254    2560 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896257    2560 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896260    2560 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896262    2560 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896264    2560 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896267    2560 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 06 17:10:28.899922 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896269    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896272    2560 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896274    2560 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896277    2560 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896279    2560 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896282    2560 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896284    2560 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896287    2560 feature_gate.go:328] unrecognized feature gate: DualReplica
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896290    2560 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896292    2560 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896295    2560 feature_gate.go:328] unrecognized feature gate: Example
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896298    2560 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896300    2560 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896303    2560 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896306    2560 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896308    2560 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896311    2560 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896314    2560 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896316    2560 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896319    2560 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 06 17:10:28.900619 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896321    2560 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 06 17:10:28.901138 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:28.896324    2560 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 06 17:10:28.901138 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.896329    2560 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 06 17:10:28.901138 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.896945    2560 server.go:962] "Client rotation is on, will bootstrap in background"
May 06 17:10:28.901138 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.899018    2560 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 06 17:10:28.901138 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.899886    2560 server.go:1019] "Starting client certificate rotation"
May 06 17:10:28.901138 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.899989    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 06 17:10:28.901138 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.900039    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 06 17:10:28.922645 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.922626    2560 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 06 17:10:28.925340 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.925311    2560 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 06 17:10:28.938826 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.938805    2560 log.go:25] "Validated CRI v1 runtime API"
May 06 17:10:28.945188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.945173    2560 log.go:25] "Validated CRI v1 image API"
May 06 17:10:28.946519 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.946498    2560 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 06 17:10:28.952038 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.952011    2560 fs.go:135] Filesystem UUIDs: map[0e1863bf-65aa-4b16-8124-d3f16ed851f7:/dev/nvme0n1p4 7B77-95E7:/dev/nvme0n1p2 da3f0801-9a59-42e6-b0b5-a7cd9927a0fd:/dev/nvme0n1p3]
May 06 17:10:28.952112 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.952037    2560 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 06 17:10:28.956750 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.956732    2560 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 06 17:10:28.957995 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.957888    2560 manager.go:217] Machine: {Timestamp:2026-05-06 17:10:28.956763062 +0000 UTC m=+0.400585500 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098941 MemoryCapacity:33164500992 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2548fda9c626f6827bc514fce696c5 SystemUUID:ec2548fd-a9c6-26f6-827b-c514fce696c5 BootID:a7202889-807f-4b2a-96b9-fb629b0e0672 Filesystems:[{Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582250496 Type:vfs Inodes:4048401 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:63:d8:b5:67:d1 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:63:d8:b5:67:d1 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:6e:9a:e6:7c:da:ae Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164500992 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 06 17:10:28.957995 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.957990    2560 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 06 17:10:28.958112 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.958074    2560 manager.go:233] Version: {KernelVersion:5.14.0-570.112.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260504-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 06 17:10:28.958449 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.958430    2560 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 06 17:10:28.958586 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.958451    2560 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-130-186.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 06 17:10:28.958637 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.958596    2560 topology_manager.go:138] "Creating topology manager with none policy"
May 06 17:10:28.958637 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.958604    2560 container_manager_linux.go:306] "Creating device plugin manager"
May 06 17:10:28.958637 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.958617    2560 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 06 17:10:28.959399 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.959387    2560 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 06 17:10:28.960294 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.960284    2560 state_mem.go:36] "Initialized new in-memory state store"
May 06 17:10:28.960398 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.960389    2560 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 06 17:10:28.962665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.962656    2560 kubelet.go:491] "Attempting to sync node with API server"
May 06 17:10:28.962703 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.962670    2560 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 06 17:10:28.962703 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.962683    2560 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 06 17:10:28.962703 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.962692    2560 kubelet.go:397] "Adding apiserver pod source"
May 06 17:10:28.962802 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.962704    2560 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 06 17:10:28.963640 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.963628    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 06 17:10:28.963698 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.963647    2560 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 06 17:10:28.966373 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.966357    2560 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 06 17:10:28.967821 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.967808    2560 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 06 17:10:28.968628 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.968612    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 06 17:10:28.968674 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.968640    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 06 17:10:28.968674 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.968646    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 06 17:10:28.968674 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.968652    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 06 17:10:28.968674 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.968658    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 06 17:10:28.968674 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.968664    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 06 17:10:28.968674 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.968671    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 06 17:10:28.968674 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.968677    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 06 17:10:28.968980 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.968685    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 06 17:10:28.968980 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.968695    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 06 17:10:28.968980 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.968705    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 06 17:10:28.969215 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.969205    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 06 17:10:28.970019 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.970010    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 06 17:10:28.970019 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.970019    2560 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 06 17:10:28.973840 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.973825    2560 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 06 17:10:28.973907 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.973864    2560 server.go:1295] "Started kubelet"
May 06 17:10:28.973947 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.973912    2560 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-130-186.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 06 17:10:28.974432 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:28.974411    2560 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-130-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 06 17:10:28.974509 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:28.974411    2560 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 06 17:10:28.974640 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.974605    2560 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 06 17:10:28.974640 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.974609    2560 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 06 17:10:28.974798 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.974666    2560 server_v1.go:47] "podresources" method="list" useActivePods=true
May 06 17:10:28.974850 ip-10-0-130-186 systemd[1]: Started Kubernetes Kubelet.
May 06 17:10:28.975813 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.975703    2560 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 06 17:10:28.977496 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.977471    2560 server.go:317] "Adding debug handlers to kubelet server"
May 06 17:10:28.980199 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:28.979348    2560 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-130-186.ec2.internal.18ad09188f92c365  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-130-186.ec2.internal,UID:ip-10-0-130-186.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-130-186.ec2.internal,},FirstTimestamp:2026-05-06 17:10:28.973839205 +0000 UTC m=+0.417661651,LastTimestamp:2026-05-06 17:10:28.973839205 +0000 UTC m=+0.417661651,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-130-186.ec2.internal,}"
May 06 17:10:28.982687 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.982667    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cn2k7"
May 06 17:10:28.984860 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.984840    2560 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 06 17:10:28.985397 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:28.985377    2560 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 06 17:10:28.985488 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.985455    2560 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 06 17:10:28.986098 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.986074    2560 volume_manager.go:295] "The desired_state_of_world populator starts"
May 06 17:10:28.986098 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.986098    2560 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 06 17:10:28.986256 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.986075    2560 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 06 17:10:28.986256 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:28.986204    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-186.ec2.internal\" not found"
May 06 17:10:28.986256 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.986218    2560 reconstruct.go:97] "Volume reconstruction finished"
May 06 17:10:28.986256 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.986229    2560 reconciler.go:26] "Reconciler: start to sync state"
May 06 17:10:28.987322 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:28.987300    2560 controller.go:145] "Failed to ensure lease exists, will retry" err="leases.coordination.k8s.io \"ip-10-0-130-186.ec2.internal\" is forbidden: User \"system:anonymous\" cannot get resource \"leases\" in API group \"coordination.k8s.io\" in the namespace \"kube-node-lease\"" interval="200ms"
May 06 17:10:28.987509 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:28.987480    2560 reflector.go:200] "Failed to watch" err="failed to list *v1.CSIDriver: csidrivers.storage.k8s.io is forbidden: User \"system:anonymous\" cannot list resource \"csidrivers\" in API group \"storage.k8s.io\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver"
May 06 17:10:28.987927 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.987906    2560 factory.go:55] Registering systemd factory
May 06 17:10:28.988012 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.987967    2560 factory.go:223] Registration of the systemd container factory successfully
May 06 17:10:28.988185 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.988173    2560 factory.go:153] Registering CRI-O factory
May 06 17:10:28.988185 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.988185    2560 factory.go:223] Registration of the crio container factory successfully
May 06 17:10:28.988302 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.988223    2560 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 06 17:10:28.988302 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.988244    2560 factory.go:103] Registering Raw factory
May 06 17:10:28.988302 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.988273    2560 manager.go:1196] Started watching for new ooms in manager
May 06 17:10:28.988984 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.988966    2560 manager.go:319] Starting recovery of all containers
May 06 17:10:28.991466 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.991439    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-cn2k7"
May 06 17:10:28.999770 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:28.999642    2560 manager.go:324] Recovery completed
May 06 17:10:29.003972 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.003958    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 06 17:10:29.006012 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.005996    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasSufficientMemory"
May 06 17:10:29.006065 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.006024    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasNoDiskPressure"
May 06 17:10:29.006065 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.006035    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasSufficientPID"
May 06 17:10:29.006548 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.006531    2560 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 06 17:10:29.006548 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.006548    2560 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 06 17:10:29.006645 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.006567    2560 state_mem.go:36] "Initialized new in-memory state store"
May 06 17:10:29.009028 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.009014    2560 policy_none.go:49] "None policy: Start"
May 06 17:10:29.009028 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.009031    2560 memory_manager.go:186] "Starting memorymanager" policy="None"
May 06 17:10:29.009142 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.009041    2560 state_mem.go:35] "Initializing new in-memory state store"
May 06 17:10:29.063764 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.063747    2560 manager.go:341] "Starting Device Plugin manager"
May 06 17:10:29.063912 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.063788    2560 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 06 17:10:29.063912 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.063798    2560 server.go:85] "Starting device plugin registration server"
May 06 17:10:29.064078 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.064065    2560 eviction_manager.go:189] "Eviction manager: starting control loop"
May 06 17:10:29.064125 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.064088    2560 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 06 17:10:29.064216 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.064197    2560 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 06 17:10:29.064305 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.064292    2560 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 06 17:10:29.064346 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.064305    2560 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 06 17:10:29.064857 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.064837    2560 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 06 17:10:29.064941 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.064872    2560 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-130-186.ec2.internal\" not found"
May 06 17:10:29.114931 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.114895    2560 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 06 17:10:29.116682 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.116660    2560 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 06 17:10:29.116772 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.116688    2560 status_manager.go:230] "Starting to sync pod status with apiserver"
May 06 17:10:29.116772 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.116705    2560 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 06 17:10:29.116772 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.116711    2560 kubelet.go:2451] "Starting kubelet main sync loop"
May 06 17:10:29.116772 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.116744    2560 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 06 17:10:29.119987 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.119969    2560 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 06 17:10:29.164502 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.164481    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 06 17:10:29.167904 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.167887    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasSufficientMemory"
May 06 17:10:29.167990 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.167916    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasNoDiskPressure"
May 06 17:10:29.167990 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.167929    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasSufficientPID"
May 06 17:10:29.167990 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.167951    2560 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-130-186.ec2.internal"
May 06 17:10:29.179193 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.179175    2560 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-130-186.ec2.internal"
May 06 17:10:29.179250 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.179197    2560 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-130-186.ec2.internal\": node \"ip-10-0-130-186.ec2.internal\" not found"
May 06 17:10:29.190432 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.190410    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-186.ec2.internal\" not found"
May 06 17:10:29.217590 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.217567    2560 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-130-186.ec2.internal"]
May 06 17:10:29.217686 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.217639    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 06 17:10:29.218537 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.218519    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasSufficientMemory"
May 06 17:10:29.218614 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.218548    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasNoDiskPressure"
May 06 17:10:29.218614 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.218557    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasSufficientPID"
May 06 17:10:29.219892 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.219879    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 06 17:10:29.220063 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.220048    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.220100 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.220078    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 06 17:10:29.220589 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.220568    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasSufficientMemory"
May 06 17:10:29.220657 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.220597    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasNoDiskPressure"
May 06 17:10:29.220657 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.220568    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasSufficientMemory"
May 06 17:10:29.220657 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.220633    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasNoDiskPressure"
May 06 17:10:29.220657 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.220645    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasSufficientPID"
May 06 17:10:29.220773 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.220612    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasSufficientPID"
May 06 17:10:29.221720 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.221709    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.221775 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.221731    2560 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 06 17:10:29.222394 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.222380    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasSufficientMemory"
May 06 17:10:29.222460 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.222408    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasNoDiskPressure"
May 06 17:10:29.222460 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.222423    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeHasSufficientPID"
May 06 17:10:29.241660 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.241641    2560 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-186.ec2.internal\" not found" node="ip-10-0-130-186.ec2.internal"
May 06 17:10:29.245657 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.245642    2560 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-130-186.ec2.internal\" not found" node="ip-10-0-130-186.ec2.internal"
May 06 17:10:29.288071 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.288045    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/906084eb4124a9dfc490db23a45f3f0b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal\" (UID: \"906084eb4124a9dfc490db23a45f3f0b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.288071 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.288073    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/906084eb4124a9dfc490db23a45f3f0b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal\" (UID: \"906084eb4124a9dfc490db23a45f3f0b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.288236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.288090    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a35c20fbb6c420ebcfcdda7c5c935059-config\") pod \"kube-apiserver-proxy-ip-10-0-130-186.ec2.internal\" (UID: \"a35c20fbb6c420ebcfcdda7c5c935059\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.291513 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.291494    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-186.ec2.internal\" not found"
May 06 17:10:29.388894 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.388857    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/906084eb4124a9dfc490db23a45f3f0b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal\" (UID: \"906084eb4124a9dfc490db23a45f3f0b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.388894 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.388891    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/906084eb4124a9dfc490db23a45f3f0b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal\" (UID: \"906084eb4124a9dfc490db23a45f3f0b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.389120 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.388909    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a35c20fbb6c420ebcfcdda7c5c935059-config\") pod \"kube-apiserver-proxy-ip-10-0-130-186.ec2.internal\" (UID: \"a35c20fbb6c420ebcfcdda7c5c935059\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.389120 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.388985    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/906084eb4124a9dfc490db23a45f3f0b-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal\" (UID: \"906084eb4124a9dfc490db23a45f3f0b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.389120 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.389027    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/a35c20fbb6c420ebcfcdda7c5c935059-config\") pod \"kube-apiserver-proxy-ip-10-0-130-186.ec2.internal\" (UID: \"a35c20fbb6c420ebcfcdda7c5c935059\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.389120 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.389067    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/906084eb4124a9dfc490db23a45f3f0b-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal\" (UID: \"906084eb4124a9dfc490db23a45f3f0b\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.391926 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.391910    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-186.ec2.internal\" not found"
May 06 17:10:29.492729 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.492651    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-186.ec2.internal\" not found"
May 06 17:10:29.544864 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.544829    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.549412 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.549392    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.593479 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.593451    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-186.ec2.internal\" not found"
May 06 17:10:29.693936 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.693893    2560 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-130-186.ec2.internal\" not found"
May 06 17:10:29.783520 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.783493    2560 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 06 17:10:29.786526 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.786510    2560 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.797518 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.797504    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 06 17:10:29.799009 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.798997    2560 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-186.ec2.internal"
May 06 17:10:29.813956 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.813936    2560 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 06 17:10:29.900240 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.900214    2560 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 06 17:10:29.900730 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.900347    2560 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 06 17:10:29.900730 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.900376    2560 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 06 17:10:29.939218 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.939197    2560 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 06 17:10:29.963059 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.963034    2560 apiserver.go:52] "Watching apiserver"
May 06 17:10:29.968836 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.968815    2560 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 06 17:10:29.970701 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.970679    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-diagnostics/network-check-target-jgnb4","kube-system/kube-apiserver-proxy-ip-10-0-130-186.ec2.internal","openshift-image-registry/node-ca-tfv78","openshift-multus/multus-66rrf","openshift-multus/network-metrics-daemon-cwlz9","openshift-network-operator/iptables-alerter-wgztz","openshift-ovn-kubernetes/ovnkube-node-9s6rp","kube-system/konnectivity-agent-sdndx","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7","openshift-cluster-node-tuning-operator/tuned-drn4b","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal","openshift-multus/multus-additional-cni-plugins-klb5m"]
May 06 17:10:29.973510 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.973492    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:29.973602 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.973572    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:29.974438 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.974421    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tfv78"
May 06 17:10:29.974580 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.974521    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-66rrf"
May 06 17:10:29.975706 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.975687    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:29.975775 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:29.975732    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:29.976374 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.976359    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 06 17:10:29.976760 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.976732    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 06 17:10:29.976760 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.976755    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 06 17:10:29.976914 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.976766    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 06 17:10:29.976914 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.976811    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 06 17:10:29.976914 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.976748    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 06 17:10:29.977057 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.976913    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 06 17:10:29.977257 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.977242    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-wnmdq\""
May 06 17:10:29.977304 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.977297    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-xf56v\""
May 06 17:10:29.978004 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.977989    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wgztz"
May 06 17:10:29.978221 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.978196    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.979385 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.979370    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sdndx"
May 06 17:10:29.979736 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.979714    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 06 17:10:29.979811 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.979801    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 06 17:10:29.980334 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.980317    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 06 17:10:29.980442 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.980368    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 06 17:10:29.980586 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.980541    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-h7f27\""
May 06 17:10:29.980655 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.980611    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 06 17:10:29.980655 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.980628    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 06 17:10:29.980752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.980614    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:29.980846 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.980753    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 06 17:10:29.981098 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.981081    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 06 17:10:29.981201 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.981120    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 06 17:10:29.981201 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.981121    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-wz2vr\""
May 06 17:10:29.981472 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.981458    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-pn98x\""
May 06 17:10:29.981472 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.981469    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 06 17:10:29.981557 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.981492    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 06 17:10:29.981818 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.981800    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.982557 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.982542    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 06 17:10:29.984197 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.983133    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 06 17:10:29.984197 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.983163    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:29.984197 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.983241    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-dfgww\""
May 06 17:10:29.984197 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.983928    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 06 17:10:29.984197 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.983943    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-2twxn\""
May 06 17:10:29.984197 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.983944    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 06 17:10:29.984569 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.984528    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 06 17:10:29.985122 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.985020    2560 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 06 17:10:29.985641 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.985622    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 06 17:10:29.985969 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.985952    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-mdbrn\""
May 06 17:10:29.986361 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.986345    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 06 17:10:29.986863 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.986850    2560 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 06 17:10:29.991482 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991447    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m4lsh\" (UniqueName: \"kubernetes.io/projected/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-kube-api-access-m4lsh\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.991558 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991492    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-registration-dir\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:29.991558 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991509    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-sysconfig\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.991558 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991523    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b28d0114-0ff1-4b91-873b-467eb3f73088-ovnkube-config\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.991698 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991564    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-cnibin\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:29.991698 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991584    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-run\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.991698 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991634    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-run-k8s-cni-cncf-io\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.991698 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991671    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-run-multus-certs\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.991834 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991700    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4q5vs\" (UniqueName: \"kubernetes.io/projected/795dd069-201d-40d2-9e08-1e1842bcc66b-kube-api-access-4q5vs\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:29.991834 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991741    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.991834 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991768    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-cni-binary-copy\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:29.991834 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991797    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-sysctl-d\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.991834 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991815    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-kubelet\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.991834 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991830    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b28d0114-0ff1-4b91-873b-467eb3f73088-ovn-node-metrics-cert\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.992077 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991844    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b28d0114-0ff1-4b91-873b-467eb3f73088-ovnkube-script-lib\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.992077 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991873    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-cni-binary-copy\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.992077 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991896    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e360f9ba-379a-45f3-9ae6-dee0e26375db-konnectivity-ca\") pod \"konnectivity-agent-sdndx\" (UID: \"e360f9ba-379a-45f3-9ae6-dee0e26375db\") " pod="kube-system/konnectivity-agent-sdndx"
May 06 17:10:29.992077 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991914    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:29.992077 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991933    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-68fg6\" (UniqueName: \"kubernetes.io/projected/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-kube-api-access-68fg6\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:29.992077 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991947    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-kubernetes\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.992077 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.991967    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-sysctl-conf\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.992077 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992022    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b4a8d487-7a1b-4c15-8ef9-5c4743b481cd-host-slash\") pod \"iptables-alerter-wgztz\" (UID: \"b4a8d487-7a1b-4c15-8ef9-5c4743b481cd\") " pod="openshift-network-operator/iptables-alerter-wgztz"
May 06 17:10:29.992077 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992059    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-systemd-units\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992088    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-run-openvswitch\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992116    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-system-cni-dir\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992140    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-var-lib-cni-bin\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992183    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e360f9ba-379a-45f3-9ae6-dee0e26375db-agent-certs\") pod \"konnectivity-agent-sdndx\" (UID: \"e360f9ba-379a-45f3-9ae6-dee0e26375db\") " pod="kube-system/konnectivity-agent-sdndx"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992206    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-var-lib-kubelet\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992229    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-etc-kubernetes\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992263    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-os-release\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992290    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992312    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-cnibin\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992333    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-os-release\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992364    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2wt6w\" (UniqueName: \"kubernetes.io/projected/24a44446-b589-49fd-a785-92e01601b8b4-kube-api-access-2wt6w\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992399    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-etc-openvswitch\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992426    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-cni-bin\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992452    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nws2n\" (UniqueName: \"kubernetes.io/projected/b28d0114-0ff1-4b91-873b-467eb3f73088-kube-api-access-nws2n\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992476    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-system-cni-dir\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:29.992500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992501    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jzkfj\" (UniqueName: \"kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj\") pod \"network-check-target-jgnb4\" (UID: \"566bd2fc-87ab-4ff0-8362-cb0a84be38de\") " pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992527    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-var-lib-cni-multus\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992563    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-modprobe-d\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992587    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b4a8d487-7a1b-4c15-8ef9-5c4743b481cd-iptables-alerter-script\") pod \"iptables-alerter-wgztz\" (UID: \"b4a8d487-7a1b-4c15-8ef9-5c4743b481cd\") " pod="openshift-network-operator/iptables-alerter-wgztz"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992612    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-slash\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992642    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-run-systemd\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992666    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-var-lib-openvswitch\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992688    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-log-socket\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992709    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61680c01-bda6-4b45-ae5b-32adc5ce27c8-host\") pod \"node-ca-tfv78\" (UID: \"61680c01-bda6-4b45-ae5b-32adc5ce27c8\") " pod="openshift-image-registry/node-ca-tfv78"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992730    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-sys\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992753    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-run-ovn-kubernetes\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992775    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b28d0114-0ff1-4b91-873b-467eb3f73088-env-overrides\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992799    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-multus-cni-dir\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992823    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-multus-socket-dir-parent\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992846    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-multus-daemon-config\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992883    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-systemd\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992918    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-host\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.993298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992945    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992971    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-multus-conf-dir\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.992994    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-device-dir\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993018    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f71e8a4-559a-4618-8f4f-10fe11a4c659-tmp\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993042    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993067    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-sys-fs\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993089    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-tuned\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993113    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p6gq7\" (UniqueName: \"kubernetes.io/projected/7f71e8a4-559a-4618-8f4f-10fe11a4c659-kube-api-access-p6gq7\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993139    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-run-ovn\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993191    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993242    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5r4nj\" (UniqueName: \"kubernetes.io/projected/b4a8d487-7a1b-4c15-8ef9-5c4743b481cd-kube-api-access-5r4nj\") pod \"iptables-alerter-wgztz\" (UID: \"b4a8d487-7a1b-4c15-8ef9-5c4743b481cd\") " pod="openshift-network-operator/iptables-alerter-wgztz"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993270    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-run-netns\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993297    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-run-netns\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993331    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-var-lib-kubelet\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993355    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-socket-dir\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993396    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-node-log\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993390    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-05 17:05:28 +0000 UTC" deadline="2027-11-02 09:50:26.310893551 +0000 UTC"
May 06 17:10:29.993954 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993430    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13072h39m56.317467378s"
May 06 17:10:29.994549 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993438    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-cni-netd\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:29.994549 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993474    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-etc-selinux\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:29.994549 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993502    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/61680c01-bda6-4b45-ae5b-32adc5ce27c8-serviceca\") pod \"node-ca-tfv78\" (UID: \"61680c01-bda6-4b45-ae5b-32adc5ce27c8\") " pod="openshift-image-registry/node-ca-tfv78"
May 06 17:10:29.994549 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993527    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csj4f\" (UniqueName: \"kubernetes.io/projected/61680c01-bda6-4b45-ae5b-32adc5ce27c8-kube-api-access-csj4f\") pod \"node-ca-tfv78\" (UID: \"61680c01-bda6-4b45-ae5b-32adc5ce27c8\") " pod="openshift-image-registry/node-ca-tfv78"
May 06 17:10:29.994549 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993551    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-hostroot\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:29.994549 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.993573    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-lib-modules\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:29.994839 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:29.994818    2560 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 06 17:10:30.024361 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.024294    2560 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-fwc8l"
May 06 17:10:30.033401 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.033378    2560 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-fwc8l"
May 06 17:10:30.093774 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093748    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-hostroot\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.093890 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093780    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-lib-modules\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.093890 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093797    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m4lsh\" (UniqueName: \"kubernetes.io/projected/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-kube-api-access-m4lsh\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.093890 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093812    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-registration-dir\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.093890 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093827    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-sysconfig\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.093890 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093846    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b28d0114-0ff1-4b91-873b-467eb3f73088-ovnkube-config\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.093890 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093853    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-hostroot\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.093890 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093868    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-cnibin\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093905    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-cnibin\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093909    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-run\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093915    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-sysconfig\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093905    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-registration-dir\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093948    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-run-k8s-cni-cncf-io\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093953    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-run\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093976    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-lib-modules\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093992    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-run-k8s-cni-cncf-io\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.093978    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-run-multus-certs\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094015    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-run-multus-certs\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094029    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4q5vs\" (UniqueName: \"kubernetes.io/projected/795dd069-201d-40d2-9e08-1e1842bcc66b-kube-api-access-4q5vs\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094058    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094087    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-cni-binary-copy\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094113    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-sysctl-d\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094133    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.094236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094138    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-kubelet\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.094756 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094290    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b28d0114-0ff1-4b91-873b-467eb3f73088-ovn-node-metrics-cert\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.094756 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094289    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-kubelet\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.094756 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094522    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-sysctl-d\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.094756 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094564    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b28d0114-0ff1-4b91-873b-467eb3f73088-ovnkube-script-lib\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.094756 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094593    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-cni-binary-copy\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.094756 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094622    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e360f9ba-379a-45f3-9ae6-dee0e26375db-konnectivity-ca\") pod \"konnectivity-agent-sdndx\" (UID: \"e360f9ba-379a-45f3-9ae6-dee0e26375db\") " pod="kube-system/konnectivity-agent-sdndx"
May 06 17:10:30.094756 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094657    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.094756 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094683    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-68fg6\" (UniqueName: \"kubernetes.io/projected/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-kube-api-access-68fg6\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.094756 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094704    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-kubernetes\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.094756 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094727    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-sysctl-conf\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.094756 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094750    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b4a8d487-7a1b-4c15-8ef9-5c4743b481cd-host-slash\") pod \"iptables-alerter-wgztz\" (UID: \"b4a8d487-7a1b-4c15-8ef9-5c4743b481cd\") " pod="openshift-network-operator/iptables-alerter-wgztz"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094777    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-systemd-units\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094788    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-cni-binary-copy\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094802    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-run-openvswitch\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094804    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-kubelet-dir\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094828    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-system-cni-dir\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094853    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-var-lib-cni-bin\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094877    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e360f9ba-379a-45f3-9ae6-dee0e26375db-agent-certs\") pod \"konnectivity-agent-sdndx\" (UID: \"e360f9ba-379a-45f3-9ae6-dee0e26375db\") " pod="kube-system/konnectivity-agent-sdndx"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094902    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-var-lib-kubelet\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094927    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-etc-kubernetes\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094951    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-os-release\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094976    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094995    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/b28d0114-0ff1-4b91-873b-467eb3f73088-ovnkube-config\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094999    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-cnibin\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095043    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-cnibin\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095055    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-os-release\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095083    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2wt6w\" (UniqueName: \"kubernetes.io/projected/24a44446-b589-49fd-a785-92e01601b8b4-kube-api-access-2wt6w\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095087    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-var-lib-cni-bin\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.095274 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095109    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-etc-openvswitch\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095135    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-cni-bin\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095177    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nws2n\" (UniqueName: \"kubernetes.io/projected/b28d0114-0ff1-4b91-873b-467eb3f73088-kube-api-access-nws2n\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095204    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-system-cni-dir\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095228    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jzkfj\" (UniqueName: \"kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj\") pod \"network-check-target-jgnb4\" (UID: \"566bd2fc-87ab-4ff0-8362-cb0a84be38de\") " pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095260    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-var-lib-cni-multus\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095287    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-modprobe-d\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095306    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-kubernetes\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095320    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-var-lib-kubelet\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095320    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b4a8d487-7a1b-4c15-8ef9-5c4743b481cd-iptables-alerter-script\") pod \"iptables-alerter-wgztz\" (UID: \"b4a8d487-7a1b-4c15-8ef9-5c4743b481cd\") " pod="openshift-network-operator/iptables-alerter-wgztz"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.094998    2560 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095375    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-etc-kubernetes\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095376    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-cni-binary-copy\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095439    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-os-release\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095462    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-sysctl-conf\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095494    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-slash\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095522    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-run-systemd\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095547    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-var-lib-openvswitch\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095552    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/e360f9ba-379a-45f3-9ae6-dee0e26375db-konnectivity-ca\") pod \"konnectivity-agent-sdndx\" (UID: \"e360f9ba-379a-45f3-9ae6-dee0e26375db\") " pod="kube-system/konnectivity-agent-sdndx"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095573    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-log-socket\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095600    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61680c01-bda6-4b45-ae5b-32adc5ce27c8-host\") pod \"node-ca-tfv78\" (UID: \"61680c01-bda6-4b45-ae5b-32adc5ce27c8\") " pod="openshift-image-registry/node-ca-tfv78"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095555    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/b28d0114-0ff1-4b91-873b-467eb3f73088-ovnkube-script-lib\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095624    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-sys\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095636    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-cni-bin\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095671    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-sys\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095687    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-run-ovn-kubernetes\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095712    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b4a8d487-7a1b-4c15-8ef9-5c4743b481cd-host-slash\") pod \"iptables-alerter-wgztz\" (UID: \"b4a8d487-7a1b-4c15-8ef9-5c4743b481cd\") " pod="openshift-network-operator/iptables-alerter-wgztz"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095717    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b28d0114-0ff1-4b91-873b-467eb3f73088-env-overrides\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095751    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-systemd-units\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095761    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-multus-cni-dir\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095774    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-system-cni-dir\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095787    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-multus-socket-dir-parent\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095827    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095831    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-os-release\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095881    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-var-lib-openvswitch\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095885    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-run-systemd\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.096901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095848    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-slash\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095927    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-log-socket\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095990    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/61680c01-bda6-4b45-ae5b-32adc5ce27c8-host\") pod \"node-ca-tfv78\" (UID: \"61680c01-bda6-4b45-ae5b-32adc5ce27c8\") " pod="openshift-image-registry/node-ca-tfv78"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.096012    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-multus-daemon-config\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.096040    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-multus-socket-dir-parent\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.096068    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-systemd\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095788    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-run-openvswitch\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.095604    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-etc-openvswitch\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.096604    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-host\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097212    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/b28d0114-0ff1-4b91-873b-467eb3f73088-env-overrides\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097225    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.096760    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-multus-cni-dir\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097120    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-multus-daemon-config\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097259    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-multus-conf-dir\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097141    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-system-cni-dir\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097285    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-device-dir\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097314    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f71e8a4-559a-4618-8f4f-10fe11a4c659-tmp\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097352    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.097739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097385    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-sys-fs\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097415    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-tuned\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097440    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p6gq7\" (UniqueName: \"kubernetes.io/projected/7f71e8a4-559a-4618-8f4f-10fe11a4c659-kube-api-access-p6gq7\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097472    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-run-ovn\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097483    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/b4a8d487-7a1b-4c15-8ef9-5c4743b481cd-iptables-alerter-script\") pod \"iptables-alerter-wgztz\" (UID: \"b4a8d487-7a1b-4c15-8ef9-5c4743b481cd\") " pod="openshift-network-operator/iptables-alerter-wgztz"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097504    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097537    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5r4nj\" (UniqueName: \"kubernetes.io/projected/b4a8d487-7a1b-4c15-8ef9-5c4743b481cd-kube-api-access-5r4nj\") pod \"iptables-alerter-wgztz\" (UID: \"b4a8d487-7a1b-4c15-8ef9-5c4743b481cd\") " pod="openshift-network-operator/iptables-alerter-wgztz"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097566    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-run-netns\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097589    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-run-netns\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097615    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-var-lib-kubelet\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097641    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-socket-dir\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097668    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-node-log\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097697    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-cni-netd\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097721    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-etc-selinux\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097751    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/61680c01-bda6-4b45-ae5b-32adc5ce27c8-serviceca\") pod \"node-ca-tfv78\" (UID: \"61680c01-bda6-4b45-ae5b-32adc5ce27c8\") " pod="openshift-image-registry/node-ca-tfv78"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.097780    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-csj4f\" (UniqueName: \"kubernetes.io/projected/61680c01-bda6-4b45-ae5b-32adc5ce27c8-kube-api-access-csj4f\") pod \"node-ca-tfv78\" (UID: \"61680c01-bda6-4b45-ae5b-32adc5ce27c8\") " pod="openshift-image-registry/node-ca-tfv78"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.096699    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-host\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.098418 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:30.098170    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:10:30.099186 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:30.098256    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs podName:24a44446-b589-49fd-a785-92e01601b8b4 nodeName:}" failed. No retries permitted until 2026-05-06 17:10:30.598214699 +0000 UTC m=+2.042037141 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs") pod "network-metrics-daemon-cwlz9" (UID: "24a44446-b589-49fd-a785-92e01601b8b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:10:30.099186 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.096803    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-systemd\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.099186 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.098489    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-multus-conf-dir\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.099186 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.098553    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-device-dir\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.099186 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.098678    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/b28d0114-0ff1-4b91-873b-467eb3f73088-ovn-node-metrics-cert\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.099186 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.098970    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/e360f9ba-379a-45f3-9ae6-dee0e26375db-agent-certs\") pod \"konnectivity-agent-sdndx\" (UID: \"e360f9ba-379a-45f3-9ae6-dee0e26375db\") " pod="kube-system/konnectivity-agent-sdndx"
May 06 17:10:30.099186 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.099058    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-var-lib-cni-multus\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.099186 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.099120    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-run-netns\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.099633 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.099293    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-node-log\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.099633 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.099378    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-run-netns\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.099633 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.099432    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-host-var-lib-kubelet\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.099633 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.099457    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-cni-netd\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.099633 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.099541    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-socket-dir\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.099633 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.099554    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-etc-selinux\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.099633 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.099624    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/795dd069-201d-40d2-9e08-1e1842bcc66b-sys-fs\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.099944 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.099931    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/61680c01-bda6-4b45-ae5b-32adc5ce27c8-serviceca\") pod \"node-ca-tfv78\" (UID: \"61680c01-bda6-4b45-ae5b-32adc5ce27c8\") " pod="openshift-image-registry/node-ca-tfv78"
May 06 17:10:30.100199 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.100175    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-tuning-conf-dir\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.100312 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.100292    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.100453 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.100436    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-host-run-ovn-kubernetes\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.100683 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.100664    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/b28d0114-0ff1-4b91-873b-467eb3f73088-run-ovn\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.100784 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.100761    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-modprobe-d\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.100844 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.100806    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/7f71e8a4-559a-4618-8f4f-10fe11a4c659-tmp\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.104260 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.104239    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/7f71e8a4-559a-4618-8f4f-10fe11a4c659-etc-tuned\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.116144 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:30.116124    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 06 17:10:30.116144 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:30.116145    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 06 17:10:30.116314 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:30.116171    2560 projected.go:194] Error preparing data for projected volume kube-api-access-jzkfj for pod openshift-network-diagnostics/network-check-target-jgnb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:10:30.116314 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:30.116218    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj podName:566bd2fc-87ab-4ff0-8362-cb0a84be38de nodeName:}" failed. No retries permitted until 2026-05-06 17:10:30.616205506 +0000 UTC m=+2.060027931 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jzkfj" (UniqueName: "kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj") pod "network-check-target-jgnb4" (UID: "566bd2fc-87ab-4ff0-8362-cb0a84be38de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:10:30.118229 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.118144    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m4lsh\" (UniqueName: \"kubernetes.io/projected/bde1dd92-05d2-474d-b20d-0bf6c96d8cfd-kube-api-access-m4lsh\") pod \"multus-66rrf\" (UID: \"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd\") " pod="openshift-multus/multus-66rrf"
May 06 17:10:30.118321 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.118267    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nws2n\" (UniqueName: \"kubernetes.io/projected/b28d0114-0ff1-4b91-873b-467eb3f73088-kube-api-access-nws2n\") pod \"ovnkube-node-9s6rp\" (UID: \"b28d0114-0ff1-4b91-873b-467eb3f73088\") " pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.118762 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.118738    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4q5vs\" (UniqueName: \"kubernetes.io/projected/795dd069-201d-40d2-9e08-1e1842bcc66b-kube-api-access-4q5vs\") pod \"aws-ebs-csi-driver-node-2z9z7\" (UID: \"795dd069-201d-40d2-9e08-1e1842bcc66b\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.118869 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.118832    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-68fg6\" (UniqueName: \"kubernetes.io/projected/8feb5920-e85e-4a4d-bcbd-52a8369c0eec-kube-api-access-68fg6\") pod \"multus-additional-cni-plugins-klb5m\" (UID: \"8feb5920-e85e-4a4d-bcbd-52a8369c0eec\") " pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.119435 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.119414    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5r4nj\" (UniqueName: \"kubernetes.io/projected/b4a8d487-7a1b-4c15-8ef9-5c4743b481cd-kube-api-access-5r4nj\") pod \"iptables-alerter-wgztz\" (UID: \"b4a8d487-7a1b-4c15-8ef9-5c4743b481cd\") " pod="openshift-network-operator/iptables-alerter-wgztz"
May 06 17:10:30.119762 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.119743    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p6gq7\" (UniqueName: \"kubernetes.io/projected/7f71e8a4-559a-4618-8f4f-10fe11a4c659-kube-api-access-p6gq7\") pod \"tuned-drn4b\" (UID: \"7f71e8a4-559a-4618-8f4f-10fe11a4c659\") " pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.120177 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.120143    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2wt6w\" (UniqueName: \"kubernetes.io/projected/24a44446-b589-49fd-a785-92e01601b8b4-kube-api-access-2wt6w\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:30.121077 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:30.121051    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod906084eb4124a9dfc490db23a45f3f0b.slice/crio-bbb00dea9d95f8831c693de0626652f436a12b39c31c6dbe1e25726b64ea7220 WatchSource:0}: Error finding container bbb00dea9d95f8831c693de0626652f436a12b39c31c6dbe1e25726b64ea7220: Status 404 returned error can't find the container with id bbb00dea9d95f8831c693de0626652f436a12b39c31c6dbe1e25726b64ea7220
May 06 17:10:30.121262 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:30.121246    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda35c20fbb6c420ebcfcdda7c5c935059.slice/crio-cadd91561194ea6914ef667a674cbef9e50d73797dc1594eae9ee49cb46a1d39 WatchSource:0}: Error finding container cadd91561194ea6914ef667a674cbef9e50d73797dc1594eae9ee49cb46a1d39: Status 404 returned error can't find the container with id cadd91561194ea6914ef667a674cbef9e50d73797dc1594eae9ee49cb46a1d39
May 06 17:10:30.122928 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.122909    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csj4f\" (UniqueName: \"kubernetes.io/projected/61680c01-bda6-4b45-ae5b-32adc5ce27c8-kube-api-access-csj4f\") pod \"node-ca-tfv78\" (UID: \"61680c01-bda6-4b45-ae5b-32adc5ce27c8\") " pod="openshift-image-registry/node-ca-tfv78"
May 06 17:10:30.127327 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.127313    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 06 17:10:30.299630 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.299553    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-tfv78"
May 06 17:10:30.305409 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:30.305385    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod61680c01_bda6_4b45_ae5b_32adc5ce27c8.slice/crio-60f5967a66d43543840dce60ad5ab209affdb5b5fdcf01becb67a0cd1c53fe06 WatchSource:0}: Error finding container 60f5967a66d43543840dce60ad5ab209affdb5b5fdcf01becb67a0cd1c53fe06: Status 404 returned error can't find the container with id 60f5967a66d43543840dce60ad5ab209affdb5b5fdcf01becb67a0cd1c53fe06
May 06 17:10:30.330655 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.330631    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-66rrf"
May 06 17:10:30.337851 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:30.337827    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbde1dd92_05d2_474d_b20d_0bf6c96d8cfd.slice/crio-0d5b9f7d2d14e449e9e786cc225ee45227f8d72d38515f83d820da2be7c3f80e WatchSource:0}: Error finding container 0d5b9f7d2d14e449e9e786cc225ee45227f8d72d38515f83d820da2be7c3f80e: Status 404 returned error can't find the container with id 0d5b9f7d2d14e449e9e786cc225ee45227f8d72d38515f83d820da2be7c3f80e
May 06 17:10:30.347461 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.347443    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-wgztz"
May 06 17:10:30.352835 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:30.352815    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4a8d487_7a1b_4c15_8ef9_5c4743b481cd.slice/crio-b4a9deb837fd350d4f34376eeebb27632a8459cc0e85d5ba24d9ece72ebd9716 WatchSource:0}: Error finding container b4a9deb837fd350d4f34376eeebb27632a8459cc0e85d5ba24d9ece72ebd9716: Status 404 returned error can't find the container with id b4a9deb837fd350d4f34376eeebb27632a8459cc0e85d5ba24d9ece72ebd9716
May 06 17:10:30.359310 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.359291    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:30.364976 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:30.364948    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb28d0114_0ff1_4b91_873b_467eb3f73088.slice/crio-bc9f4d7db18de1ab5f726b4fc59f3570d989702c12663fd652714566264654c9 WatchSource:0}: Error finding container bc9f4d7db18de1ab5f726b4fc59f3570d989702c12663fd652714566264654c9: Status 404 returned error can't find the container with id bc9f4d7db18de1ab5f726b4fc59f3570d989702c12663fd652714566264654c9
May 06 17:10:30.365083 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.364995    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-sdndx"
May 06 17:10:30.371650 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:30.371629    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode360f9ba_379a_45f3_9ae6_dee0e26375db.slice/crio-4a87b4380f8f7c2940a6b7d0a3ee7f1df599b4acab2f1a35520de0ebddd7615d WatchSource:0}: Error finding container 4a87b4380f8f7c2940a6b7d0a3ee7f1df599b4acab2f1a35520de0ebddd7615d: Status 404 returned error can't find the container with id 4a87b4380f8f7c2940a6b7d0a3ee7f1df599b4acab2f1a35520de0ebddd7615d
May 06 17:10:30.376225 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.376211    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7"
May 06 17:10:30.382099 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:30.382076    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod795dd069_201d_40d2_9e08_1e1842bcc66b.slice/crio-697941d1c070a687fda79fa4a331ea9101b88bfa403cbc39909ee3438a99239c WatchSource:0}: Error finding container 697941d1c070a687fda79fa4a331ea9101b88bfa403cbc39909ee3438a99239c: Status 404 returned error can't find the container with id 697941d1c070a687fda79fa4a331ea9101b88bfa403cbc39909ee3438a99239c
May 06 17:10:30.393677 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.393658    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-drn4b"
May 06 17:10:30.400882 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:30.400547    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7f71e8a4_559a_4618_8f4f_10fe11a4c659.slice/crio-f285f8296ad1ed8ce9550990eef10b37e801fd0b749206cfb5b565536e312050 WatchSource:0}: Error finding container f285f8296ad1ed8ce9550990eef10b37e801fd0b749206cfb5b565536e312050: Status 404 returned error can't find the container with id f285f8296ad1ed8ce9550990eef10b37e801fd0b749206cfb5b565536e312050
May 06 17:10:30.411003 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.410984    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-klb5m"
May 06 17:10:30.416592 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:10:30.416565    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8feb5920_e85e_4a4d_bcbd_52a8369c0eec.slice/crio-445b5a64a026df8fadbe214255c18c918831e7aec29b566c91eb474ffe6166ec WatchSource:0}: Error finding container 445b5a64a026df8fadbe214255c18c918831e7aec29b566c91eb474ffe6166ec: Status 404 returned error can't find the container with id 445b5a64a026df8fadbe214255c18c918831e7aec29b566c91eb474ffe6166ec
May 06 17:10:30.508291 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.508263    2560 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 06 17:10:30.601997 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.601917    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:30.602175 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:30.602047    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:10:30.602175 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:30.602106    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs podName:24a44446-b589-49fd-a785-92e01601b8b4 nodeName:}" failed. No retries permitted until 2026-05-06 17:10:31.602087771 +0000 UTC m=+3.045910200 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs") pod "network-metrics-daemon-cwlz9" (UID: "24a44446-b589-49fd-a785-92e01601b8b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:10:30.703091 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.703051    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jzkfj\" (UniqueName: \"kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj\") pod \"network-check-target-jgnb4\" (UID: \"566bd2fc-87ab-4ff0-8362-cb0a84be38de\") " pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:30.703291 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:30.703271    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 06 17:10:30.703379 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:30.703300    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 06 17:10:30.703379 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:30.703313    2560 projected.go:194] Error preparing data for projected volume kube-api-access-jzkfj for pod openshift-network-diagnostics/network-check-target-jgnb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:10:30.703542 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:30.703396    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj podName:566bd2fc-87ab-4ff0-8362-cb0a84be38de nodeName:}" failed. No retries permitted until 2026-05-06 17:10:31.703363552 +0000 UTC m=+3.147185996 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzkfj" (UniqueName: "kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj") pod "network-check-target-jgnb4" (UID: "566bd2fc-87ab-4ff0-8362-cb0a84be38de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:10:30.977779 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:30.977751    2560 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 06 17:10:31.035029 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.034985    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-05 17:05:30 +0000 UTC" deadline="2027-10-04 22:46:58.666842122 +0000 UTC"
May 06 17:10:31.035029 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.035026    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12389h36m27.631820154s"
May 06 17:10:31.128182 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.128004    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-66rrf" event={"ID":"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd","Type":"ContainerStarted","Data":"0d5b9f7d2d14e449e9e786cc225ee45227f8d72d38515f83d820da2be7c3f80e"}
May 06 17:10:31.134568 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.131989    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-drn4b" event={"ID":"7f71e8a4-559a-4618-8f4f-10fe11a4c659","Type":"ContainerStarted","Data":"f285f8296ad1ed8ce9550990eef10b37e801fd0b749206cfb5b565536e312050"}
May 06 17:10:31.135950 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.135372    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7" event={"ID":"795dd069-201d-40d2-9e08-1e1842bcc66b","Type":"ContainerStarted","Data":"697941d1c070a687fda79fa4a331ea9101b88bfa403cbc39909ee3438a99239c"}
May 06 17:10:31.140393 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.140361    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tfv78" event={"ID":"61680c01-bda6-4b45-ae5b-32adc5ce27c8","Type":"ContainerStarted","Data":"60f5967a66d43543840dce60ad5ab209affdb5b5fdcf01becb67a0cd1c53fe06"}
May 06 17:10:31.150261 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.150220    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-186.ec2.internal" event={"ID":"a35c20fbb6c420ebcfcdda7c5c935059","Type":"ContainerStarted","Data":"cadd91561194ea6914ef667a674cbef9e50d73797dc1594eae9ee49cb46a1d39"}
May 06 17:10:31.157578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.157523    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal" event={"ID":"906084eb4124a9dfc490db23a45f3f0b","Type":"ContainerStarted","Data":"bbb00dea9d95f8831c693de0626652f436a12b39c31c6dbe1e25726b64ea7220"}
May 06 17:10:31.166530 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.166501    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-klb5m" event={"ID":"8feb5920-e85e-4a4d-bcbd-52a8369c0eec","Type":"ContainerStarted","Data":"445b5a64a026df8fadbe214255c18c918831e7aec29b566c91eb474ffe6166ec"}
May 06 17:10:31.172786 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.172742    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sdndx" event={"ID":"e360f9ba-379a-45f3-9ae6-dee0e26375db","Type":"ContainerStarted","Data":"4a87b4380f8f7c2940a6b7d0a3ee7f1df599b4acab2f1a35520de0ebddd7615d"}
May 06 17:10:31.179278 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.179242    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp" event={"ID":"b28d0114-0ff1-4b91-873b-467eb3f73088","Type":"ContainerStarted","Data":"bc9f4d7db18de1ab5f726b4fc59f3570d989702c12663fd652714566264654c9"}
May 06 17:10:31.182720 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.182675    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wgztz" event={"ID":"b4a8d487-7a1b-4c15-8ef9-5c4743b481cd","Type":"ContainerStarted","Data":"b4a9deb837fd350d4f34376eeebb27632a8459cc0e85d5ba24d9ece72ebd9716"}
May 06 17:10:31.609731 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.609104    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:31.609731 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:31.609288    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:10:31.609731 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:31.609354    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs podName:24a44446-b589-49fd-a785-92e01601b8b4 nodeName:}" failed. No retries permitted until 2026-05-06 17:10:33.609334502 +0000 UTC m=+5.053156931 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs") pod "network-metrics-daemon-cwlz9" (UID: "24a44446-b589-49fd-a785-92e01601b8b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:10:31.710476 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:31.710438    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jzkfj\" (UniqueName: \"kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj\") pod \"network-check-target-jgnb4\" (UID: \"566bd2fc-87ab-4ff0-8362-cb0a84be38de\") " pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:31.710654 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:31.710611    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 06 17:10:31.710654 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:31.710632    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 06 17:10:31.710654 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:31.710645    2560 projected.go:194] Error preparing data for projected volume kube-api-access-jzkfj for pod openshift-network-diagnostics/network-check-target-jgnb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:10:31.710817 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:31.710700    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj podName:566bd2fc-87ab-4ff0-8362-cb0a84be38de nodeName:}" failed. No retries permitted until 2026-05-06 17:10:33.710683453 +0000 UTC m=+5.154505884 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzkfj" (UniqueName: "kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj") pod "network-check-target-jgnb4" (UID: "566bd2fc-87ab-4ff0-8362-cb0a84be38de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:10:32.035705 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:32.035620    2560 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-05 17:05:30 +0000 UTC" deadline="2027-10-30 00:42:08.835736578 +0000 UTC"
May 06 17:10:32.035705 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:32.035658    2560 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12991h31m36.800082365s"
May 06 17:10:32.116974 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:32.116944    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:32.117169 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:32.117071    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:32.117528 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:32.117508    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:32.117632 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:32.117613    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:32.878063 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:32.878036    2560 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 06 17:10:33.627393 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:33.627355    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:33.627842 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:33.627537    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:10:33.627842 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:33.627603    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs podName:24a44446-b589-49fd-a785-92e01601b8b4 nodeName:}" failed. No retries permitted until 2026-05-06 17:10:37.627584983 +0000 UTC m=+9.071407423 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs") pod "network-metrics-daemon-cwlz9" (UID: "24a44446-b589-49fd-a785-92e01601b8b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:10:33.728223 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:33.728181    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jzkfj\" (UniqueName: \"kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj\") pod \"network-check-target-jgnb4\" (UID: \"566bd2fc-87ab-4ff0-8362-cb0a84be38de\") " pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:33.728400 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:33.728378    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 06 17:10:33.728400 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:33.728400    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 06 17:10:33.728517 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:33.728414    2560 projected.go:194] Error preparing data for projected volume kube-api-access-jzkfj for pod openshift-network-diagnostics/network-check-target-jgnb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:10:33.728517 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:33.728475    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj podName:566bd2fc-87ab-4ff0-8362-cb0a84be38de nodeName:}" failed. No retries permitted until 2026-05-06 17:10:37.728457083 +0000 UTC m=+9.172279524 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzkfj" (UniqueName: "kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj") pod "network-check-target-jgnb4" (UID: "566bd2fc-87ab-4ff0-8362-cb0a84be38de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:10:33.979553 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:33.979511    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-4nq4b"]
May 06 17:10:33.985503 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:33.985476    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4nq4b"
May 06 17:10:33.987478 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:33.987454    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-rvt4b\""
May 06 17:10:33.987823 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:33.987774    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 06 17:10:33.987823 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:33.987778    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 06 17:10:34.030162 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:34.029890    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3378859b-bdd9-4a9f-98cf-dba6a953b3f7-hosts-file\") pod \"node-resolver-4nq4b\" (UID: \"3378859b-bdd9-4a9f-98cf-dba6a953b3f7\") " pod="openshift-dns/node-resolver-4nq4b"
May 06 17:10:34.030162 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:34.029938    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wnggw\" (UniqueName: \"kubernetes.io/projected/3378859b-bdd9-4a9f-98cf-dba6a953b3f7-kube-api-access-wnggw\") pod \"node-resolver-4nq4b\" (UID: \"3378859b-bdd9-4a9f-98cf-dba6a953b3f7\") " pod="openshift-dns/node-resolver-4nq4b"
May 06 17:10:34.030162 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:34.030045    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3378859b-bdd9-4a9f-98cf-dba6a953b3f7-tmp-dir\") pod \"node-resolver-4nq4b\" (UID: \"3378859b-bdd9-4a9f-98cf-dba6a953b3f7\") " pod="openshift-dns/node-resolver-4nq4b"
May 06 17:10:34.117963 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:34.117889    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:34.118117 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:34.118041    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:34.118212 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:34.118119    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:34.118279 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:34.118261    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:34.132235 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:34.131407    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3378859b-bdd9-4a9f-98cf-dba6a953b3f7-hosts-file\") pod \"node-resolver-4nq4b\" (UID: \"3378859b-bdd9-4a9f-98cf-dba6a953b3f7\") " pod="openshift-dns/node-resolver-4nq4b"
May 06 17:10:34.132235 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:34.131452    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wnggw\" (UniqueName: \"kubernetes.io/projected/3378859b-bdd9-4a9f-98cf-dba6a953b3f7-kube-api-access-wnggw\") pod \"node-resolver-4nq4b\" (UID: \"3378859b-bdd9-4a9f-98cf-dba6a953b3f7\") " pod="openshift-dns/node-resolver-4nq4b"
May 06 17:10:34.132235 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:34.131518    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3378859b-bdd9-4a9f-98cf-dba6a953b3f7-tmp-dir\") pod \"node-resolver-4nq4b\" (UID: \"3378859b-bdd9-4a9f-98cf-dba6a953b3f7\") " pod="openshift-dns/node-resolver-4nq4b"
May 06 17:10:34.132235 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:34.131868    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/3378859b-bdd9-4a9f-98cf-dba6a953b3f7-tmp-dir\") pod \"node-resolver-4nq4b\" (UID: \"3378859b-bdd9-4a9f-98cf-dba6a953b3f7\") " pod="openshift-dns/node-resolver-4nq4b"
May 06 17:10:34.132235 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:34.131944    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/3378859b-bdd9-4a9f-98cf-dba6a953b3f7-hosts-file\") pod \"node-resolver-4nq4b\" (UID: \"3378859b-bdd9-4a9f-98cf-dba6a953b3f7\") " pod="openshift-dns/node-resolver-4nq4b"
May 06 17:10:34.142067 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:34.141856    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wnggw\" (UniqueName: \"kubernetes.io/projected/3378859b-bdd9-4a9f-98cf-dba6a953b3f7-kube-api-access-wnggw\") pod \"node-resolver-4nq4b\" (UID: \"3378859b-bdd9-4a9f-98cf-dba6a953b3f7\") " pod="openshift-dns/node-resolver-4nq4b"
May 06 17:10:34.297788 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:34.297419    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-4nq4b"
May 06 17:10:36.117217 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:36.117177    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:36.117685 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:36.117322    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:36.117685 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:36.117500    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:36.117685 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:36.117597    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:37.660629 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:37.660595    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:37.661071 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:37.660723    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:10:37.661071 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:37.660786    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs podName:24a44446-b589-49fd-a785-92e01601b8b4 nodeName:}" failed. No retries permitted until 2026-05-06 17:10:45.660766297 +0000 UTC m=+17.104588728 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs") pod "network-metrics-daemon-cwlz9" (UID: "24a44446-b589-49fd-a785-92e01601b8b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:10:37.761605 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:37.761570    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jzkfj\" (UniqueName: \"kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj\") pod \"network-check-target-jgnb4\" (UID: \"566bd2fc-87ab-4ff0-8362-cb0a84be38de\") " pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:37.761766 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:37.761743    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 06 17:10:37.761766 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:37.761765    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 06 17:10:37.761888 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:37.761778    2560 projected.go:194] Error preparing data for projected volume kube-api-access-jzkfj for pod openshift-network-diagnostics/network-check-target-jgnb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:10:37.761888 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:37.761833    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj podName:566bd2fc-87ab-4ff0-8362-cb0a84be38de nodeName:}" failed. No retries permitted until 2026-05-06 17:10:45.761817365 +0000 UTC m=+17.205639791 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzkfj" (UniqueName: "kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj") pod "network-check-target-jgnb4" (UID: "566bd2fc-87ab-4ff0-8362-cb0a84be38de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:10:38.118528 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:38.117857    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:38.118528 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:38.118002    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:38.118528 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:38.118395    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:38.118528 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:38.118480    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:40.117958 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:40.117921    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:40.118449 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:40.117931    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:40.118449 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:40.118046    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:40.118449 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:40.118138    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:42.117530 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:42.117492    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:42.118003 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:42.117501    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:42.118003 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:42.117604    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:42.118003 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:42.117692    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:44.116967 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:44.116930    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:44.117388 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:44.116941    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:44.117388 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:44.117052    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:44.117388 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:44.117121    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:45.713309 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:45.713085    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:45.713797 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:45.713241    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:10:45.713797 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:45.713483    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs podName:24a44446-b589-49fd-a785-92e01601b8b4 nodeName:}" failed. No retries permitted until 2026-05-06 17:11:01.713457368 +0000 UTC m=+33.157279806 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs") pod "network-metrics-daemon-cwlz9" (UID: "24a44446-b589-49fd-a785-92e01601b8b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:10:45.814670 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:45.814635    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jzkfj\" (UniqueName: \"kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj\") pod \"network-check-target-jgnb4\" (UID: \"566bd2fc-87ab-4ff0-8362-cb0a84be38de\") " pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:45.814863 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:45.814817    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 06 17:10:45.814863 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:45.814843    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 06 17:10:45.814863 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:45.814858    2560 projected.go:194] Error preparing data for projected volume kube-api-access-jzkfj for pod openshift-network-diagnostics/network-check-target-jgnb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:10:45.815019 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:45.814914    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj podName:566bd2fc-87ab-4ff0-8362-cb0a84be38de nodeName:}" failed. No retries permitted until 2026-05-06 17:11:01.814900517 +0000 UTC m=+33.258722942 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzkfj" (UniqueName: "kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj") pod "network-check-target-jgnb4" (UID: "566bd2fc-87ab-4ff0-8362-cb0a84be38de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:10:46.117079 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:46.116993    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:46.117259 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:46.116993    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:46.117259 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:46.117146    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:46.117259 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:46.117233    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:48.117091 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:48.117056    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:48.117529 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:48.117056    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:48.117529 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:48.117197    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:48.117529 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:48.117256    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:49.220463 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:49.219814    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-186.ec2.internal" event={"ID":"a35c20fbb6c420ebcfcdda7c5c935059","Type":"ContainerStarted","Data":"e058185b36a150165d2564085135e4f2c56f86b5907f5667fb7ac3ab5005c8d7"}
May 06 17:10:49.224688 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:49.224658    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4nq4b" event={"ID":"3378859b-bdd9-4a9f-98cf-dba6a953b3f7","Type":"ContainerStarted","Data":"0bb2f5abf65af0de9bf9e58091841b1caa0a138813eb04d9f8865723aee2321c"}
May 06 17:10:49.229501 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:49.229477    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-drn4b" event={"ID":"7f71e8a4-559a-4618-8f4f-10fe11a4c659","Type":"ContainerStarted","Data":"7954a1af437495c22a9cfb9438a58ba419274ee01d7918949ca74f8d2800d5d1"}
May 06 17:10:49.254234 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:49.254182    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-drn4b" podStartSLOduration=1.772901091 podStartE2EDuration="20.254138526s" podCreationTimestamp="2026-05-06 17:10:29 +0000 UTC" firstStartedPulling="2026-05-06 17:10:30.402221222 +0000 UTC m=+1.846043647" lastFinishedPulling="2026-05-06 17:10:48.883458642 +0000 UTC m=+20.327281082" observedRunningTime="2026-05-06 17:10:49.253851437 +0000 UTC m=+20.697673883" watchObservedRunningTime="2026-05-06 17:10:49.254138526 +0000 UTC m=+20.697961030"
May 06 17:10:49.254438 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:49.254419    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-130-186.ec2.internal" podStartSLOduration=20.254414031 podStartE2EDuration="20.254414031s" podCreationTimestamp="2026-05-06 17:10:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-06 17:10:49.237255311 +0000 UTC m=+20.681077759" watchObservedRunningTime="2026-05-06 17:10:49.254414031 +0000 UTC m=+20.698236480"
May 06 17:10:50.117081 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.116905    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:50.117188 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.116910    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:50.117225 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:50.117191    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:50.117306 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:50.117284    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:50.232033 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.232007    2560 generic.go:358] "Generic (PLEG): container finished" podID="8feb5920-e85e-4a4d-bcbd-52a8369c0eec" containerID="2a279b2885d93acd179d4c42a700451012c50b667516820ed76051085156e49a" exitCode=0
May 06 17:10:50.232724 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.232088    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-klb5m" event={"ID":"8feb5920-e85e-4a4d-bcbd-52a8369c0eec","Type":"ContainerDied","Data":"2a279b2885d93acd179d4c42a700451012c50b667516820ed76051085156e49a"}
May 06 17:10:50.233229 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.233206    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-sdndx" event={"ID":"e360f9ba-379a-45f3-9ae6-dee0e26375db","Type":"ContainerStarted","Data":"2220a514e95af1a8cbbf0750bb0dec8ccaa5a28537c85e5e9fdbbebf417c7d93"}
May 06 17:10:50.235632 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.235610    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp" event={"ID":"b28d0114-0ff1-4b91-873b-467eb3f73088","Type":"ContainerStarted","Data":"0b64a91ca849766af7a6e335992211b9f3c36e5bd3ad794fca6e347c2c25f7e6"}
May 06 17:10:50.235728 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.235636    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp" event={"ID":"b28d0114-0ff1-4b91-873b-467eb3f73088","Type":"ContainerStarted","Data":"95c820a68d26ca724ca75ba722d63b9435c60b136fa249e63bcde7b697eb57b2"}
May 06 17:10:50.235728 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.235650    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp" event={"ID":"b28d0114-0ff1-4b91-873b-467eb3f73088","Type":"ContainerStarted","Data":"982b5167f0d31296af22acba686db676869e8d0bf1e1e3dee59f0f34931cbcac"}
May 06 17:10:50.235728 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.235664    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp" event={"ID":"b28d0114-0ff1-4b91-873b-467eb3f73088","Type":"ContainerStarted","Data":"def8b75084ab16904a3c3ff2746f71abaf9b93c31fb4374aed3bbd1f1363069b"}
May 06 17:10:50.235728 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.235676    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp" event={"ID":"b28d0114-0ff1-4b91-873b-467eb3f73088","Type":"ContainerStarted","Data":"4a1a47b97e69cdfe93c47d5da0254ecf809620a904b717b235e1adeb893febc4"}
May 06 17:10:50.236817 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.236798    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-66rrf" event={"ID":"bde1dd92-05d2-474d-b20d-0bf6c96d8cfd","Type":"ContainerStarted","Data":"558381d9d8b081f2a047be23d022da3a1e5182804b886091b0eabf42e1c83960"}
May 06 17:10:50.238085 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.238066    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7" event={"ID":"795dd069-201d-40d2-9e08-1e1842bcc66b","Type":"ContainerStarted","Data":"0a5c0c3f762bb783f9cf2818ee89121b930351626be0d36f41a8e2a393e77aa2"}
May 06 17:10:50.239096 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.239076    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-tfv78" event={"ID":"61680c01-bda6-4b45-ae5b-32adc5ce27c8","Type":"ContainerStarted","Data":"18030ea2b63f301edd076726f694ec8fe743f087e2fc7d3bce029869635b6441"}
May 06 17:10:50.240293 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.240271    2560 generic.go:358] "Generic (PLEG): container finished" podID="906084eb4124a9dfc490db23a45f3f0b" containerID="fcc575f6f2598229949231147e3790834b33d4f9d5d61f9da133e51166ef8603" exitCode=0
May 06 17:10:50.240386 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.240325    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal" event={"ID":"906084eb4124a9dfc490db23a45f3f0b","Type":"ContainerDied","Data":"fcc575f6f2598229949231147e3790834b33d4f9d5d61f9da133e51166ef8603"}
May 06 17:10:50.241521 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.241500    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-4nq4b" event={"ID":"3378859b-bdd9-4a9f-98cf-dba6a953b3f7","Type":"ContainerStarted","Data":"689b290e78225602dd3085131f06189d9b523f0b45e21a53b5368d913fe7e4bb"}
May 06 17:10:50.285013 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.284958    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-tfv78" podStartSLOduration=2.7106786659999997 podStartE2EDuration="21.284939338s" podCreationTimestamp="2026-05-06 17:10:29 +0000 UTC" firstStartedPulling="2026-05-06 17:10:30.30694703 +0000 UTC m=+1.750769454" lastFinishedPulling="2026-05-06 17:10:48.881207686 +0000 UTC m=+20.325030126" observedRunningTime="2026-05-06 17:10:50.284481544 +0000 UTC m=+21.728303992" watchObservedRunningTime="2026-05-06 17:10:50.284939338 +0000 UTC m=+21.728761785"
May 06 17:10:50.303285 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.303234    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-66rrf" podStartSLOduration=2.72534138 podStartE2EDuration="21.303214187s" podCreationTimestamp="2026-05-06 17:10:29 +0000 UTC" firstStartedPulling="2026-05-06 17:10:30.339425763 +0000 UTC m=+1.783248189" lastFinishedPulling="2026-05-06 17:10:48.91729857 +0000 UTC m=+20.361120996" observedRunningTime="2026-05-06 17:10:50.302678679 +0000 UTC m=+21.746501133" watchObservedRunningTime="2026-05-06 17:10:50.303214187 +0000 UTC m=+21.747036635"
May 06 17:10:50.323498 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.323438    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-sdndx" podStartSLOduration=2.815113908 podStartE2EDuration="21.323419675s" podCreationTimestamp="2026-05-06 17:10:29 +0000 UTC" firstStartedPulling="2026-05-06 17:10:30.372899462 +0000 UTC m=+1.816721887" lastFinishedPulling="2026-05-06 17:10:48.881205215 +0000 UTC m=+20.325027654" observedRunningTime="2026-05-06 17:10:50.32265342 +0000 UTC m=+21.766475880" watchObservedRunningTime="2026-05-06 17:10:50.323419675 +0000 UTC m=+21.767242122"
May 06 17:10:50.337720 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:50.337678    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-4nq4b" podStartSLOduration=17.337662996 podStartE2EDuration="17.337662996s" podCreationTimestamp="2026-05-06 17:10:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-06 17:10:50.337263864 +0000 UTC m=+21.781086326" watchObservedRunningTime="2026-05-06 17:10:50.337662996 +0000 UTC m=+21.781485442"
May 06 17:10:51.137984 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:51.137955    2560 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 06 17:10:51.245116 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:51.245078    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-wgztz" event={"ID":"b4a8d487-7a1b-4c15-8ef9-5c4743b481cd","Type":"ContainerStarted","Data":"85133308491eb569783afafdd369d5e75f2b7dac6bdcb19e1588d986a8741760"}
May 06 17:10:51.247013 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:51.246950    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7" event={"ID":"795dd069-201d-40d2-9e08-1e1842bcc66b","Type":"ContainerStarted","Data":"8c565c3dc9daba7abc4605f993a4cbf9b34ffe61e459491c735156213d099974"}
May 06 17:10:51.248705 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:51.248685    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal" event={"ID":"906084eb4124a9dfc490db23a45f3f0b","Type":"ContainerStarted","Data":"befe0cd103ca2cce4ef0c9b139222fd32e770d309c4f524db5b475cf950f9046"}
May 06 17:10:51.251612 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:51.251569    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp" event={"ID":"b28d0114-0ff1-4b91-873b-467eb3f73088","Type":"ContainerStarted","Data":"8701139756a2c0dbb4574ba8f6a2a7e15016cea4f25cec111e11965ffcde4685"}
May 06 17:10:51.259780 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:51.259744    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-wgztz" podStartSLOduration=3.7327105720000002 podStartE2EDuration="22.259731583s" podCreationTimestamp="2026-05-06 17:10:29 +0000 UTC" firstStartedPulling="2026-05-06 17:10:30.354185474 +0000 UTC m=+1.798007899" lastFinishedPulling="2026-05-06 17:10:48.881206483 +0000 UTC m=+20.325028910" observedRunningTime="2026-05-06 17:10:51.259590958 +0000 UTC m=+22.703413404" watchObservedRunningTime="2026-05-06 17:10:51.259731583 +0000 UTC m=+22.703554030"
May 06 17:10:52.078817 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:52.078711    2560 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-06T17:10:51.137976973Z","UUID":"c0a1a446-b405-4fb4-b9b9-61ea75bcee86","Handler":null,"Name":"","Endpoint":""}
May 06 17:10:52.081543 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:52.081517    2560 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 06 17:10:52.081686 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:52.081552    2560 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 06 17:10:52.117903 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:52.117876    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:52.118057 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:52.118001    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:52.118115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:52.118055    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:52.118196 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:52.118176    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:52.204194 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:52.204141    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-sdndx"
May 06 17:10:52.204830 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:52.204809    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-sdndx"
May 06 17:10:52.219382 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:52.219334    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-130-186.ec2.internal" podStartSLOduration=23.219321108 podStartE2EDuration="23.219321108s" podCreationTimestamp="2026-05-06 17:10:29 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-06 17:10:51.273261423 +0000 UTC m=+22.717083875" watchObservedRunningTime="2026-05-06 17:10:52.219321108 +0000 UTC m=+23.663143554"
May 06 17:10:52.254513 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:52.254393    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-sdndx"
May 06 17:10:52.254910 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:52.254537    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-sdndx"
May 06 17:10:53.257745 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:53.257707    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7" event={"ID":"795dd069-201d-40d2-9e08-1e1842bcc66b","Type":"ContainerStarted","Data":"7b8d75a70d1e6cc4aae5978f46556cf13ee32b36535ee318fd33a10bdb85aec3"}
May 06 17:10:53.261261 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:53.261225    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp" event={"ID":"b28d0114-0ff1-4b91-873b-467eb3f73088","Type":"ContainerStarted","Data":"3fc642137bccdb9ad54faa82654e56f0968940bcf36d3e947b4999d54ea268cb"}
May 06 17:10:53.275124 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:53.275080    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-2z9z7" podStartSLOduration=2.215120402 podStartE2EDuration="24.275068572s" podCreationTimestamp="2026-05-06 17:10:29 +0000 UTC" firstStartedPulling="2026-05-06 17:10:30.383070832 +0000 UTC m=+1.826893257" lastFinishedPulling="2026-05-06 17:10:52.443018995 +0000 UTC m=+23.886841427" observedRunningTime="2026-05-06 17:10:53.274758485 +0000 UTC m=+24.718580932" watchObservedRunningTime="2026-05-06 17:10:53.275068572 +0000 UTC m=+24.718891022"
May 06 17:10:54.116998 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:54.116964    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:54.117204 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:54.116970    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:54.117204 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:54.117092    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:54.117204 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:54.117184    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:55.270058 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:55.269875    2560 generic.go:358] "Generic (PLEG): container finished" podID="8feb5920-e85e-4a4d-bcbd-52a8369c0eec" containerID="e832f15bc20e1442d2253af0e23b07b76563dd1ecbf40e85a9718ebcd8d05691" exitCode=0
May 06 17:10:55.270901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:55.269957    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-klb5m" event={"ID":"8feb5920-e85e-4a4d-bcbd-52a8369c0eec","Type":"ContainerDied","Data":"e832f15bc20e1442d2253af0e23b07b76563dd1ecbf40e85a9718ebcd8d05691"}
May 06 17:10:55.273402 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:55.273376    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp" event={"ID":"b28d0114-0ff1-4b91-873b-467eb3f73088","Type":"ContainerStarted","Data":"7cc6b1565b195f1618c72290d5d6b632e8c13fdb2f0da7aad5bf1eb1daef84b3"}
May 06 17:10:55.273684 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:55.273663    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:55.288864 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:55.288843    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:56.117775 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:56.117748    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:56.117907 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:56.117748    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:56.117907 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:56.117854    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:56.117987 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:56.117924    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:56.275709 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:56.275652    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:56.275709 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:56.275691    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:56.293489 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:56.293324    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:10:56.320036 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:56.319954    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp" podStartSLOduration=8.18806971 podStartE2EDuration="27.319934556s" podCreationTimestamp="2026-05-06 17:10:29 +0000 UTC" firstStartedPulling="2026-05-06 17:10:30.366678768 +0000 UTC m=+1.810501196" lastFinishedPulling="2026-05-06 17:10:49.498543614 +0000 UTC m=+20.942366042" observedRunningTime="2026-05-06 17:10:55.317789732 +0000 UTC m=+26.761612179" watchObservedRunningTime="2026-05-06 17:10:56.319934556 +0000 UTC m=+27.763757004"
May 06 17:10:56.835797 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:56.835767    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jgnb4"]
May 06 17:10:56.835957 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:56.835866    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:56.835957 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:56.835948    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:56.840085 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:56.840056    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cwlz9"]
May 06 17:10:56.840245 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:56.840204    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:56.840338 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:56.840314    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:10:57.277929 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:57.277898    2560 generic.go:358] "Generic (PLEG): container finished" podID="8feb5920-e85e-4a4d-bcbd-52a8369c0eec" containerID="c262556d54a4a162e5ae7f96392f2146aa3c61231412d162e31fccee251884fd" exitCode=0
May 06 17:10:57.278396 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:57.277988    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-klb5m" event={"ID":"8feb5920-e85e-4a4d-bcbd-52a8369c0eec","Type":"ContainerDied","Data":"c262556d54a4a162e5ae7f96392f2146aa3c61231412d162e31fccee251884fd"}
May 06 17:10:58.117479 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:58.117449    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:10:58.117601 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:58.117559    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:10:58.281415 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:58.281327    2560 generic.go:358] "Generic (PLEG): container finished" podID="8feb5920-e85e-4a4d-bcbd-52a8369c0eec" containerID="8e07e3eb419d4633cd0f80b6ed8dd1824326d9be928782cc92f336d82351ff97" exitCode=0
May 06 17:10:58.281756 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:58.281417    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-klb5m" event={"ID":"8feb5920-e85e-4a4d-bcbd-52a8369c0eec","Type":"ContainerDied","Data":"8e07e3eb419d4633cd0f80b6ed8dd1824326d9be928782cc92f336d82351ff97"}
May 06 17:10:59.118285 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:10:59.118254    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:10:59.118415 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:10:59.118381    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:11:00.117249 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:00.117214    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:11:00.117738 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:00.117343    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:11:01.117941 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:01.117864    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:11:01.118342 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:01.118017    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-cwlz9" podUID="24a44446-b589-49fd-a785-92e01601b8b4"
May 06 17:11:01.225556 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:01.225527    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4nq4b_3378859b-bdd9-4a9f-98cf-dba6a953b3f7/dns-node-resolver/0.log"
May 06 17:11:01.727633 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:01.727591    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:11:01.727851 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:01.727732    2560 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:11:01.727851 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:01.727798    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs podName:24a44446-b589-49fd-a785-92e01601b8b4 nodeName:}" failed. No retries permitted until 2026-05-06 17:11:33.727779274 +0000 UTC m=+65.171601699 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs") pod "network-metrics-daemon-cwlz9" (UID: "24a44446-b589-49fd-a785-92e01601b8b4") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 06 17:11:01.828329 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:01.828285    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jzkfj\" (UniqueName: \"kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj\") pod \"network-check-target-jgnb4\" (UID: \"566bd2fc-87ab-4ff0-8362-cb0a84be38de\") " pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:11:01.828532 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:01.828452    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 06 17:11:01.828532 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:01.828479    2560 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 06 17:11:01.828532 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:01.828496    2560 projected.go:194] Error preparing data for projected volume kube-api-access-jzkfj for pod openshift-network-diagnostics/network-check-target-jgnb4: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:11:01.828674 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:01.828560    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj podName:566bd2fc-87ab-4ff0-8362-cb0a84be38de nodeName:}" failed. No retries permitted until 2026-05-06 17:11:33.828537561 +0000 UTC m=+65.272360020 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jzkfj" (UniqueName: "kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj") pod "network-check-target-jgnb4" (UID: "566bd2fc-87ab-4ff0-8362-cb0a84be38de") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 06 17:11:02.117500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.117427    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:11:02.117736 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:02.117535    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-jgnb4" podUID="566bd2fc-87ab-4ff0-8362-cb0a84be38de"
May 06 17:11:02.406967 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.406676    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tfv78_61680c01-bda6-4b45-ae5b-32adc5ce27c8/node-ca/0.log"
May 06 17:11:02.845961 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.845932    2560 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-130-186.ec2.internal" event="NodeReady"
May 06 17:11:02.846125 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.846076    2560 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 06 17:11:02.895259 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.895229    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-thrt5"]
May 06 17:11:02.930457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.930430    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-2b4zh"]
May 06 17:11:02.930634 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.930576    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-thrt5"
May 06 17:11:02.935015 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.934989    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 06 17:11:02.935214 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.934991    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 06 17:11:02.935324 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.935305    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-hfv2w\""
May 06 17:11:02.948607 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.948589    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-thrt5"]
May 06 17:11:02.948714 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.948613    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2b4zh"]
May 06 17:11:02.948772 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.948711    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:02.951218 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.951197    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 06 17:11:02.951319 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.951243    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 06 17:11:02.951383 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.951243    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 06 17:11:02.951383 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:02.951343    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-m5w6t\""
May 06 17:11:03.036424 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.036339    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf1db626-38d5-434d-a7ec-d48493c2605b-tmp-dir\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:03.036424 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.036395    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v449c\" (UniqueName: \"kubernetes.io/projected/995b0633-d889-4d0e-ab20-1ab11cef7e8d-kube-api-access-v449c\") pod \"ingress-canary-2b4zh\" (UID: \"995b0633-d889-4d0e-ab20-1ab11cef7e8d\") " pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:03.036649 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.036474    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:03.036649 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.036518    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert\") pod \"ingress-canary-2b4zh\" (UID: \"995b0633-d889-4d0e-ab20-1ab11cef7e8d\") " pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:03.036649 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.036570    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xdqqj\" (UniqueName: \"kubernetes.io/projected/bf1db626-38d5-434d-a7ec-d48493c2605b-kube-api-access-xdqqj\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:03.036649 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.036634    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bf1db626-38d5-434d-a7ec-d48493c2605b-config-volume\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:03.117896 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.117853    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:11:03.120428 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.120398    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 06 17:11:03.120713 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.120689    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7hgjs\""
May 06 17:11:03.137575 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.137549    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xdqqj\" (UniqueName: \"kubernetes.io/projected/bf1db626-38d5-434d-a7ec-d48493c2605b-kube-api-access-xdqqj\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:03.137693 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.137595    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bf1db626-38d5-434d-a7ec-d48493c2605b-config-volume\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:03.137693 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.137629    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf1db626-38d5-434d-a7ec-d48493c2605b-tmp-dir\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:03.137693 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.137654    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v449c\" (UniqueName: \"kubernetes.io/projected/995b0633-d889-4d0e-ab20-1ab11cef7e8d-kube-api-access-v449c\") pod \"ingress-canary-2b4zh\" (UID: \"995b0633-d889-4d0e-ab20-1ab11cef7e8d\") " pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:03.137841 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.137710    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:03.137841 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.137745    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert\") pod \"ingress-canary-2b4zh\" (UID: \"995b0633-d889-4d0e-ab20-1ab11cef7e8d\") " pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:03.137841 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:03.137837    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 06 17:11:03.137995 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:03.137901    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert podName:995b0633-d889-4d0e-ab20-1ab11cef7e8d nodeName:}" failed. No retries permitted until 2026-05-06 17:11:03.637883275 +0000 UTC m=+35.081705716 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert") pod "ingress-canary-2b4zh" (UID: "995b0633-d889-4d0e-ab20-1ab11cef7e8d") : secret "canary-serving-cert" not found
May 06 17:11:03.138060 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:03.138024    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 06 17:11:03.138113 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:03.138063    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls podName:bf1db626-38d5-434d-a7ec-d48493c2605b nodeName:}" failed. No retries permitted until 2026-05-06 17:11:03.638050037 +0000 UTC m=+35.081872475 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls") pod "dns-default-thrt5" (UID: "bf1db626-38d5-434d-a7ec-d48493c2605b") : secret "dns-default-metrics-tls" not found
May 06 17:11:03.138323 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.138291    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/bf1db626-38d5-434d-a7ec-d48493c2605b-config-volume\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:03.138415 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.138355    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/bf1db626-38d5-434d-a7ec-d48493c2605b-tmp-dir\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:03.151227 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.151203    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v449c\" (UniqueName: \"kubernetes.io/projected/995b0633-d889-4d0e-ab20-1ab11cef7e8d-kube-api-access-v449c\") pod \"ingress-canary-2b4zh\" (UID: \"995b0633-d889-4d0e-ab20-1ab11cef7e8d\") " pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:03.160401 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.160371    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xdqqj\" (UniqueName: \"kubernetes.io/projected/bf1db626-38d5-434d-a7ec-d48493c2605b-kube-api-access-xdqqj\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:03.299676 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.299599    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-7fd896b478-qb428"]
May 06 17:11:03.329830 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.329800    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7fd896b478-qb428"]
May 06 17:11:03.329991 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.329925    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7fd896b478-qb428"
May 06 17:11:03.332586 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.332560    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
May 06 17:11:03.332745 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.332679    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
May 06 17:11:03.332745 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.332691    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
May 06 17:11:03.332877 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.332793    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-7vpmd\""
May 06 17:11:03.332975 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.332953    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
May 06 17:11:03.440254 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.440213    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0c0144e6-044f-4fcc-81f2-70ce99cc3c32-signing-key\") pod \"service-ca-7fd896b478-qb428\" (UID: \"0c0144e6-044f-4fcc-81f2-70ce99cc3c32\") " pod="openshift-service-ca/service-ca-7fd896b478-qb428"
May 06 17:11:03.440793 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.440290    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v5gtj\" (UniqueName: \"kubernetes.io/projected/0c0144e6-044f-4fcc-81f2-70ce99cc3c32-kube-api-access-v5gtj\") pod \"service-ca-7fd896b478-qb428\" (UID: \"0c0144e6-044f-4fcc-81f2-70ce99cc3c32\") " pod="openshift-service-ca/service-ca-7fd896b478-qb428"
May 06 17:11:03.440793 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.440378    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0c0144e6-044f-4fcc-81f2-70ce99cc3c32-signing-cabundle\") pod \"service-ca-7fd896b478-qb428\" (UID: \"0c0144e6-044f-4fcc-81f2-70ce99cc3c32\") " pod="openshift-service-ca/service-ca-7fd896b478-qb428"
May 06 17:11:03.543884 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.541861    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0c0144e6-044f-4fcc-81f2-70ce99cc3c32-signing-cabundle\") pod \"service-ca-7fd896b478-qb428\" (UID: \"0c0144e6-044f-4fcc-81f2-70ce99cc3c32\") " pod="openshift-service-ca/service-ca-7fd896b478-qb428"
May 06 17:11:03.543884 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.541986    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0c0144e6-044f-4fcc-81f2-70ce99cc3c32-signing-key\") pod \"service-ca-7fd896b478-qb428\" (UID: \"0c0144e6-044f-4fcc-81f2-70ce99cc3c32\") " pod="openshift-service-ca/service-ca-7fd896b478-qb428"
May 06 17:11:03.543884 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.542051    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v5gtj\" (UniqueName: \"kubernetes.io/projected/0c0144e6-044f-4fcc-81f2-70ce99cc3c32-kube-api-access-v5gtj\") pod \"service-ca-7fd896b478-qb428\" (UID: \"0c0144e6-044f-4fcc-81f2-70ce99cc3c32\") " pod="openshift-service-ca/service-ca-7fd896b478-qb428"
May 06 17:11:03.543884 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.543255    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/0c0144e6-044f-4fcc-81f2-70ce99cc3c32-signing-cabundle\") pod \"service-ca-7fd896b478-qb428\" (UID: \"0c0144e6-044f-4fcc-81f2-70ce99cc3c32\") " pod="openshift-service-ca/service-ca-7fd896b478-qb428"
May 06 17:11:03.546329 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.546305    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/0c0144e6-044f-4fcc-81f2-70ce99cc3c32-signing-key\") pod \"service-ca-7fd896b478-qb428\" (UID: \"0c0144e6-044f-4fcc-81f2-70ce99cc3c32\") " pod="openshift-service-ca/service-ca-7fd896b478-qb428"
May 06 17:11:03.550348 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.550300    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v5gtj\" (UniqueName: \"kubernetes.io/projected/0c0144e6-044f-4fcc-81f2-70ce99cc3c32-kube-api-access-v5gtj\") pod \"service-ca-7fd896b478-qb428\" (UID: \"0c0144e6-044f-4fcc-81f2-70ce99cc3c32\") " pod="openshift-service-ca/service-ca-7fd896b478-qb428"
May 06 17:11:03.640420 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.640379    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-7fd896b478-qb428"
May 06 17:11:03.643403 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.643369    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:03.643500 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.643423    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert\") pod \"ingress-canary-2b4zh\" (UID: \"995b0633-d889-4d0e-ab20-1ab11cef7e8d\") " pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:03.643597 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:03.643510    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 06 17:11:03.643597 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:03.643548    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 06 17:11:03.643597 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:03.643592    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls podName:bf1db626-38d5-434d-a7ec-d48493c2605b nodeName:}" failed. No retries permitted until 2026-05-06 17:11:04.643571343 +0000 UTC m=+36.087393769 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls") pod "dns-default-thrt5" (UID: "bf1db626-38d5-434d-a7ec-d48493c2605b") : secret "dns-default-metrics-tls" not found
May 06 17:11:03.643727 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:03.643612    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert podName:995b0633-d889-4d0e-ab20-1ab11cef7e8d nodeName:}" failed. No retries permitted until 2026-05-06 17:11:04.643602405 +0000 UTC m=+36.087424834 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert") pod "ingress-canary-2b4zh" (UID: "995b0633-d889-4d0e-ab20-1ab11cef7e8d") : secret "canary-serving-cert" not found
May 06 17:11:03.996410 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:03.996381    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-7fd896b478-qb428"]
May 06 17:11:04.082146 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:11:04.082108    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c0144e6_044f_4fcc_81f2_70ce99cc3c32.slice/crio-2a59baa3c3fb054cd74fa22eebe5be95790e883d232de7192694bcd6514a20dc WatchSource:0}: Error finding container 2a59baa3c3fb054cd74fa22eebe5be95790e883d232de7192694bcd6514a20dc: Status 404 returned error can't find the container with id 2a59baa3c3fb054cd74fa22eebe5be95790e883d232de7192694bcd6514a20dc
May 06 17:11:04.117589 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:04.117564    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:11:04.120348 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:04.120325    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c658w\""
May 06 17:11:04.120348 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:04.120338    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 06 17:11:04.120488 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:04.120356    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 06 17:11:04.295299 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:04.295249    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-klb5m" event={"ID":"8feb5920-e85e-4a4d-bcbd-52a8369c0eec","Type":"ContainerStarted","Data":"d91ad8426f631eb9a490c7955f9ad65ce47892687bf300d801fdbd7002cb49be"}
May 06 17:11:04.296327 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:04.296300    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7fd896b478-qb428" event={"ID":"0c0144e6-044f-4fcc-81f2-70ce99cc3c32","Type":"ContainerStarted","Data":"2a59baa3c3fb054cd74fa22eebe5be95790e883d232de7192694bcd6514a20dc"}
May 06 17:11:04.652582 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:04.652500    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:04.652582 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:04.652546    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert\") pod \"ingress-canary-2b4zh\" (UID: \"995b0633-d889-4d0e-ab20-1ab11cef7e8d\") " pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:04.653243 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:04.652637    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 06 17:11:04.653243 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:04.652643    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 06 17:11:04.653243 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:04.652698    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert podName:995b0633-d889-4d0e-ab20-1ab11cef7e8d nodeName:}" failed. No retries permitted until 2026-05-06 17:11:06.652685313 +0000 UTC m=+38.096507741 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert") pod "ingress-canary-2b4zh" (UID: "995b0633-d889-4d0e-ab20-1ab11cef7e8d") : secret "canary-serving-cert" not found
May 06 17:11:04.653243 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:04.652709    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls podName:bf1db626-38d5-434d-a7ec-d48493c2605b nodeName:}" failed. No retries permitted until 2026-05-06 17:11:06.652703661 +0000 UTC m=+38.096526085 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls") pod "dns-default-thrt5" (UID: "bf1db626-38d5-434d-a7ec-d48493c2605b") : secret "dns-default-metrics-tls" not found
May 06 17:11:05.300692 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:05.300659    2560 generic.go:358] "Generic (PLEG): container finished" podID="8feb5920-e85e-4a4d-bcbd-52a8369c0eec" containerID="d91ad8426f631eb9a490c7955f9ad65ce47892687bf300d801fdbd7002cb49be" exitCode=0
May 06 17:11:05.300873 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:05.300698    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-klb5m" event={"ID":"8feb5920-e85e-4a4d-bcbd-52a8369c0eec","Type":"ContainerDied","Data":"d91ad8426f631eb9a490c7955f9ad65ce47892687bf300d801fdbd7002cb49be"}
May 06 17:11:06.305369 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:06.305345    2560 generic.go:358] "Generic (PLEG): container finished" podID="8feb5920-e85e-4a4d-bcbd-52a8369c0eec" containerID="3a60246094d3a7574380822cfc7d73063869c9879024d6c48518cd487f957f6d" exitCode=0
May 06 17:11:06.305632 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:06.305379    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-klb5m" event={"ID":"8feb5920-e85e-4a4d-bcbd-52a8369c0eec","Type":"ContainerDied","Data":"3a60246094d3a7574380822cfc7d73063869c9879024d6c48518cd487f957f6d"}
May 06 17:11:06.667595 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:06.667560    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:06.667740 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:06.667623    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert\") pod \"ingress-canary-2b4zh\" (UID: \"995b0633-d889-4d0e-ab20-1ab11cef7e8d\") " pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:06.667740 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:06.667710    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 06 17:11:06.667805 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:06.667764    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 06 17:11:06.667805 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:06.667779    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls podName:bf1db626-38d5-434d-a7ec-d48493c2605b nodeName:}" failed. No retries permitted until 2026-05-06 17:11:10.667760718 +0000 UTC m=+42.111583145 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls") pod "dns-default-thrt5" (UID: "bf1db626-38d5-434d-a7ec-d48493c2605b") : secret "dns-default-metrics-tls" not found
May 06 17:11:06.667870 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:06.667819    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert podName:995b0633-d889-4d0e-ab20-1ab11cef7e8d nodeName:}" failed. No retries permitted until 2026-05-06 17:11:10.667810444 +0000 UTC m=+42.111632868 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert") pod "ingress-canary-2b4zh" (UID: "995b0633-d889-4d0e-ab20-1ab11cef7e8d") : secret "canary-serving-cert" not found
May 06 17:11:07.308791 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:07.308753    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-7fd896b478-qb428" event={"ID":"0c0144e6-044f-4fcc-81f2-70ce99cc3c32","Type":"ContainerStarted","Data":"b6e15c5296a1c77f23105c55ccd4c163e98f8021624a1abda7264562eeffef60"}
May 06 17:11:07.312093 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:07.312057    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-klb5m" event={"ID":"8feb5920-e85e-4a4d-bcbd-52a8369c0eec","Type":"ContainerStarted","Data":"31acc94108e70631dc6f4fac235de4ca32314da9dfe080dcd0e35bf6a40e1f30"}
May 06 17:11:07.327503 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:07.327457    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-7fd896b478-qb428" podStartSLOduration=2.150600142 podStartE2EDuration="4.327441318s" podCreationTimestamp="2026-05-06 17:11:03 +0000 UTC" firstStartedPulling="2026-05-06 17:11:04.087435917 +0000 UTC m=+35.531258341" lastFinishedPulling="2026-05-06 17:11:06.264277084 +0000 UTC m=+37.708099517" observedRunningTime="2026-05-06 17:11:07.326100095 +0000 UTC m=+38.769922565" watchObservedRunningTime="2026-05-06 17:11:07.327441318 +0000 UTC m=+38.771263766"
May 06 17:11:07.357989 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:07.357927    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-klb5m" podStartSLOduration=4.666212468 podStartE2EDuration="38.357908769s" podCreationTimestamp="2026-05-06 17:10:29 +0000 UTC" firstStartedPulling="2026-05-06 17:10:30.417912922 +0000 UTC m=+1.861735351" lastFinishedPulling="2026-05-06 17:11:04.109609224 +0000 UTC m=+35.553431652" observedRunningTime="2026-05-06 17:11:07.357274912 +0000 UTC m=+38.801097362" watchObservedRunningTime="2026-05-06 17:11:07.357908769 +0000 UTC m=+38.801731217"
May 06 17:11:10.700807 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:10.700764    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:10.701288 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:10.700842    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert\") pod \"ingress-canary-2b4zh\" (UID: \"995b0633-d889-4d0e-ab20-1ab11cef7e8d\") " pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:10.701288 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:10.700991    2560 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 06 17:11:10.701288 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:10.701056    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert podName:995b0633-d889-4d0e-ab20-1ab11cef7e8d nodeName:}" failed. No retries permitted until 2026-05-06 17:11:18.701031316 +0000 UTC m=+50.144853760 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert") pod "ingress-canary-2b4zh" (UID: "995b0633-d889-4d0e-ab20-1ab11cef7e8d") : secret "canary-serving-cert" not found
May 06 17:11:10.701288 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:10.701135    2560 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 06 17:11:10.701288 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:10.701191    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls podName:bf1db626-38d5-434d-a7ec-d48493c2605b nodeName:}" failed. No retries permitted until 2026-05-06 17:11:18.701173043 +0000 UTC m=+50.144995469 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls") pod "dns-default-thrt5" (UID: "bf1db626-38d5-434d-a7ec-d48493c2605b") : secret "dns-default-metrics-tls" not found
May 06 17:11:18.763036 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:18.762993    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:18.763595 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:18.763047    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert\") pod \"ingress-canary-2b4zh\" (UID: \"995b0633-d889-4d0e-ab20-1ab11cef7e8d\") " pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:18.766791 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:18.766765    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/bf1db626-38d5-434d-a7ec-d48493c2605b-metrics-tls\") pod \"dns-default-thrt5\" (UID: \"bf1db626-38d5-434d-a7ec-d48493c2605b\") " pod="openshift-dns/dns-default-thrt5"
May 06 17:11:18.766909 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:18.766888    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/995b0633-d889-4d0e-ab20-1ab11cef7e8d-cert\") pod \"ingress-canary-2b4zh\" (UID: \"995b0633-d889-4d0e-ab20-1ab11cef7e8d\") " pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:18.842623 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:18.842590    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-thrt5"
May 06 17:11:18.858358 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:18.858325    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-2b4zh"
May 06 17:11:18.992816 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:18.992781    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-thrt5"]
May 06 17:11:18.996142 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:11:18.996113    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbf1db626_38d5_434d_a7ec_d48493c2605b.slice/crio-2d7e830b47f0c6883e8c33331f8292966c9ba780f2b87f92bf5cbadaf518a54f WatchSource:0}: Error finding container 2d7e830b47f0c6883e8c33331f8292966c9ba780f2b87f92bf5cbadaf518a54f: Status 404 returned error can't find the container with id 2d7e830b47f0c6883e8c33331f8292966c9ba780f2b87f92bf5cbadaf518a54f
May 06 17:11:19.005369 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:19.005346    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-2b4zh"]
May 06 17:11:19.008929 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:11:19.008909    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995b0633_d889_4d0e_ab20_1ab11cef7e8d.slice/crio-850dbf3e832b0dd2a9e62417fb61e32b47106270732833dcb838569996665100 WatchSource:0}: Error finding container 850dbf3e832b0dd2a9e62417fb61e32b47106270732833dcb838569996665100: Status 404 returned error can't find the container with id 850dbf3e832b0dd2a9e62417fb61e32b47106270732833dcb838569996665100
May 06 17:11:19.334666 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:19.334622    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2b4zh" event={"ID":"995b0633-d889-4d0e-ab20-1ab11cef7e8d","Type":"ContainerStarted","Data":"850dbf3e832b0dd2a9e62417fb61e32b47106270732833dcb838569996665100"}
May 06 17:11:19.335524 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:19.335498    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-thrt5" event={"ID":"bf1db626-38d5-434d-a7ec-d48493c2605b","Type":"ContainerStarted","Data":"2d7e830b47f0c6883e8c33331f8292966c9ba780f2b87f92bf5cbadaf518a54f"}
May 06 17:11:21.340940 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:21.340908    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-2b4zh" event={"ID":"995b0633-d889-4d0e-ab20-1ab11cef7e8d","Type":"ContainerStarted","Data":"588d6506957263ef7a91ebcb435d7fcb05e08f9f93df3bdcbf21b1888745b978"}
May 06 17:11:21.342477 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:21.342454    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-thrt5" event={"ID":"bf1db626-38d5-434d-a7ec-d48493c2605b","Type":"ContainerStarted","Data":"b77c6359128334c6f13f7739f0fb1c530c0ce0fdaf7332a8aa089f648c39288f"}
May 06 17:11:21.342579 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:21.342482    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-thrt5" event={"ID":"bf1db626-38d5-434d-a7ec-d48493c2605b","Type":"ContainerStarted","Data":"091a3b5c9338c4c833aafb7692348534667d2fc9740fcfc29c14a16150edf817"}
May 06 17:11:21.342635 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:21.342602    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-thrt5"
May 06 17:11:21.372617 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:21.372576    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-2b4zh" podStartSLOduration=17.629503362 podStartE2EDuration="19.372562864s" podCreationTimestamp="2026-05-06 17:11:02 +0000 UTC" firstStartedPulling="2026-05-06 17:11:19.010858896 +0000 UTC m=+50.454681321" lastFinishedPulling="2026-05-06 17:11:20.753918398 +0000 UTC m=+52.197740823" observedRunningTime="2026-05-06 17:11:21.372112534 +0000 UTC m=+52.815934983" watchObservedRunningTime="2026-05-06 17:11:21.372562864 +0000 UTC m=+52.816385311"
May 06 17:11:21.401585 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:21.400001    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-thrt5" podStartSLOduration=17.646989718 podStartE2EDuration="19.399976957s" podCreationTimestamp="2026-05-06 17:11:02 +0000 UTC" firstStartedPulling="2026-05-06 17:11:18.997865466 +0000 UTC m=+50.441687891" lastFinishedPulling="2026-05-06 17:11:20.7508527 +0000 UTC m=+52.194675130" observedRunningTime="2026-05-06 17:11:21.399089051 +0000 UTC m=+52.842911495" watchObservedRunningTime="2026-05-06 17:11:21.399976957 +0000 UTC m=+52.843799405"
May 06 17:11:22.024562 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.024531    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-866jg"]
May 06 17:11:22.027501 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.027484    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.030011 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.029984    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-nrl57\""
May 06 17:11:22.030663 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.030636    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 06 17:11:22.030768 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.030730    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 06 17:11:22.030837 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.030761    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 06 17:11:22.030837 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.030788    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 06 17:11:22.047613 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.047586    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-866jg"]
May 06 17:11:22.127197 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.127164    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-c8c4474bc-h4vnh"]
May 06 17:11:22.130013 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.129992    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c8c4474bc-h4vnh"
May 06 17:11:22.133654 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.133634    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 06 17:11:22.133815 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.133797    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 06 17:11:22.134019 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.134002    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-fskx9\""
May 06 17:11:22.146081 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.146059    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c8c4474bc-h4vnh"]
May 06 17:11:22.190168 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.190129    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d19c0060-6e95-4256-a8db-eeebee71e3a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.190333 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.190194    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d19c0060-6e95-4256-a8db-eeebee71e3a6-crio-socket\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.190333 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.190225    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jx888\" (UniqueName: \"kubernetes.io/projected/d19c0060-6e95-4256-a8db-eeebee71e3a6-kube-api-access-jx888\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.190333 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.190248    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d19c0060-6e95-4256-a8db-eeebee71e3a6-data-volume\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.190333 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.190305    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d19c0060-6e95-4256-a8db-eeebee71e3a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.227043 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.227008    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-5b6fb99794-56z5g"]
May 06 17:11:22.229841 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.229822    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.231901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.231884    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-7cpjk\""
May 06 17:11:22.232304 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.232287    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 06 17:11:22.232495 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.232478    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 06 17:11:22.232754 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.232739    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 06 17:11:22.237601 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.237582    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 06 17:11:22.245341 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.245318    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b6fb99794-56z5g"]
May 06 17:11:22.290900 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.290804    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d19c0060-6e95-4256-a8db-eeebee71e3a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.290900 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.290843    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d19c0060-6e95-4256-a8db-eeebee71e3a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.290900 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.290886    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d19c0060-6e95-4256-a8db-eeebee71e3a6-crio-socket\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.291193 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.291033    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjf6g\" (UniqueName: \"kubernetes.io/projected/50600190-c8d0-4091-ad42-1655c5bb4763-kube-api-access-gjf6g\") pod \"downloads-c8c4474bc-h4vnh\" (UID: \"50600190-c8d0-4091-ad42-1655c5bb4763\") " pod="openshift-console/downloads-c8c4474bc-h4vnh"
May 06 17:11:22.291193 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.291079    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/d19c0060-6e95-4256-a8db-eeebee71e3a6-crio-socket\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.291193 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.291088    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jx888\" (UniqueName: \"kubernetes.io/projected/d19c0060-6e95-4256-a8db-eeebee71e3a6-kube-api-access-jx888\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.291193 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.291173    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d19c0060-6e95-4256-a8db-eeebee71e3a6-data-volume\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.291482 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.291464    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/d19c0060-6e95-4256-a8db-eeebee71e3a6-data-volume\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.291482 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.291475    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/d19c0060-6e95-4256-a8db-eeebee71e3a6-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.293270 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.293256    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/d19c0060-6e95-4256-a8db-eeebee71e3a6-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.299825 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.299799    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jx888\" (UniqueName: \"kubernetes.io/projected/d19c0060-6e95-4256-a8db-eeebee71e3a6-kube-api-access-jx888\") pod \"insights-runtime-extractor-866jg\" (UID: \"d19c0060-6e95-4256-a8db-eeebee71e3a6\") " pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.336655 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.336630    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-866jg"
May 06 17:11:22.391782 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.391702    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e1921d6f-0f8e-4e1e-8207-e305837a82da-registry-certificates\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.392356 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.392331    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e1921d6f-0f8e-4e1e-8207-e305837a82da-trusted-ca\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.392562 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.392544    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e1921d6f-0f8e-4e1e-8207-e305837a82da-image-registry-private-configuration\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.392738 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.392721    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6lgrl\" (UniqueName: \"kubernetes.io/projected/e1921d6f-0f8e-4e1e-8207-e305837a82da-kube-api-access-6lgrl\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.392991 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.392973    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gjf6g\" (UniqueName: \"kubernetes.io/projected/50600190-c8d0-4091-ad42-1655c5bb4763-kube-api-access-gjf6g\") pod \"downloads-c8c4474bc-h4vnh\" (UID: \"50600190-c8d0-4091-ad42-1655c5bb4763\") " pod="openshift-console/downloads-c8c4474bc-h4vnh"
May 06 17:11:22.393517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.393335    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e1921d6f-0f8e-4e1e-8207-e305837a82da-registry-tls\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.393517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.393368    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e1921d6f-0f8e-4e1e-8207-e305837a82da-installation-pull-secrets\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.393517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.393399    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e1921d6f-0f8e-4e1e-8207-e305837a82da-ca-trust-extracted\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.393517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.393422    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1921d6f-0f8e-4e1e-8207-e305837a82da-bound-sa-token\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.401353 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.401331    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjf6g\" (UniqueName: \"kubernetes.io/projected/50600190-c8d0-4091-ad42-1655c5bb4763-kube-api-access-gjf6g\") pod \"downloads-c8c4474bc-h4vnh\" (UID: \"50600190-c8d0-4091-ad42-1655c5bb4763\") " pod="openshift-console/downloads-c8c4474bc-h4vnh"
May 06 17:11:22.438306 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.438272    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-c8c4474bc-h4vnh"
May 06 17:11:22.460209 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.460175    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-866jg"]
May 06 17:11:22.463096 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:11:22.463065    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd19c0060_6e95_4256_a8db_eeebee71e3a6.slice/crio-298f7988f1dea0422c2ae10d492f163ecaeac8b1ce12b03122c16db406524c07 WatchSource:0}: Error finding container 298f7988f1dea0422c2ae10d492f163ecaeac8b1ce12b03122c16db406524c07: Status 404 returned error can't find the container with id 298f7988f1dea0422c2ae10d492f163ecaeac8b1ce12b03122c16db406524c07
May 06 17:11:22.495908 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.495729    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e1921d6f-0f8e-4e1e-8207-e305837a82da-registry-tls\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.495908 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.495771    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e1921d6f-0f8e-4e1e-8207-e305837a82da-installation-pull-secrets\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.495908 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.495802    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e1921d6f-0f8e-4e1e-8207-e305837a82da-ca-trust-extracted\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.495908 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.495829    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1921d6f-0f8e-4e1e-8207-e305837a82da-bound-sa-token\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.495908 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.495872    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e1921d6f-0f8e-4e1e-8207-e305837a82da-registry-certificates\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.496345 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.495929    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e1921d6f-0f8e-4e1e-8207-e305837a82da-trusted-ca\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.496345 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.495977    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e1921d6f-0f8e-4e1e-8207-e305837a82da-image-registry-private-configuration\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.496345 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.496005    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6lgrl\" (UniqueName: \"kubernetes.io/projected/e1921d6f-0f8e-4e1e-8207-e305837a82da-kube-api-access-6lgrl\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.498061 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.498014    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/e1921d6f-0f8e-4e1e-8207-e305837a82da-registry-certificates\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.498321 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.498278    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/e1921d6f-0f8e-4e1e-8207-e305837a82da-ca-trust-extracted\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.498665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.498620    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/e1921d6f-0f8e-4e1e-8207-e305837a82da-trusted-ca\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.499338 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.499081    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/e1921d6f-0f8e-4e1e-8207-e305837a82da-registry-tls\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.500231 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.500195    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/e1921d6f-0f8e-4e1e-8207-e305837a82da-image-registry-private-configuration\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.500437 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.500416    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/e1921d6f-0f8e-4e1e-8207-e305837a82da-installation-pull-secrets\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.527409 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.527379    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/e1921d6f-0f8e-4e1e-8207-e305837a82da-bound-sa-token\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.529903 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.529856    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6lgrl\" (UniqueName: \"kubernetes.io/projected/e1921d6f-0f8e-4e1e-8207-e305837a82da-kube-api-access-6lgrl\") pod \"image-registry-5b6fb99794-56z5g\" (UID: \"e1921d6f-0f8e-4e1e-8207-e305837a82da\") " pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.538916 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.538888    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:22.589736 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.589482    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-c8c4474bc-h4vnh"]
May 06 17:11:22.594370 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:11:22.594338    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50600190_c8d0_4091_ad42_1655c5bb4763.slice/crio-35ac8c399b075b0d97b9ced034a422381dd0616542ecea98deaaa4a2eb900b7b WatchSource:0}: Error finding container 35ac8c399b075b0d97b9ced034a422381dd0616542ecea98deaaa4a2eb900b7b: Status 404 returned error can't find the container with id 35ac8c399b075b0d97b9ced034a422381dd0616542ecea98deaaa4a2eb900b7b
May 06 17:11:22.669507 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:22.669473    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-5b6fb99794-56z5g"]
May 06 17:11:22.672835 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:11:22.672803    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode1921d6f_0f8e_4e1e_8207_e305837a82da.slice/crio-538d97189597508e0c056f6d0bcc653e385477e506018bcacdc74404d1badcb2 WatchSource:0}: Error finding container 538d97189597508e0c056f6d0bcc653e385477e506018bcacdc74404d1badcb2: Status 404 returned error can't find the container with id 538d97189597508e0c056f6d0bcc653e385477e506018bcacdc74404d1badcb2
May 06 17:11:23.350352 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:23.350316    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b6fb99794-56z5g" event={"ID":"e1921d6f-0f8e-4e1e-8207-e305837a82da","Type":"ContainerStarted","Data":"4d102fe68513ecfae1b35b40f565dafcf681f950416c4792c2abdadfc4f9cc4b"}
May 06 17:11:23.350352 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:23.350359    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-5b6fb99794-56z5g" event={"ID":"e1921d6f-0f8e-4e1e-8207-e305837a82da","Type":"ContainerStarted","Data":"538d97189597508e0c056f6d0bcc653e385477e506018bcacdc74404d1badcb2"}
May 06 17:11:23.350839 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:23.350813    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:23.351842 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:23.351817    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c8c4474bc-h4vnh" event={"ID":"50600190-c8d0-4091-ad42-1655c5bb4763","Type":"ContainerStarted","Data":"35ac8c399b075b0d97b9ced034a422381dd0616542ecea98deaaa4a2eb900b7b"}
May 06 17:11:23.353497 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:23.353468    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-866jg" event={"ID":"d19c0060-6e95-4256-a8db-eeebee71e3a6","Type":"ContainerStarted","Data":"0b3adfe666d2012b8090bc0b0a5e8817d642e8390c52c999a9baac92b7937c35"}
May 06 17:11:23.353579 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:23.353500    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-866jg" event={"ID":"d19c0060-6e95-4256-a8db-eeebee71e3a6","Type":"ContainerStarted","Data":"632752d8204fb5afcd3e05bf274064c7243460bc8114aedfac33b2fa8767ed6b"}
May 06 17:11:23.353579 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:23.353515    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-866jg" event={"ID":"d19c0060-6e95-4256-a8db-eeebee71e3a6","Type":"ContainerStarted","Data":"298f7988f1dea0422c2ae10d492f163ecaeac8b1ce12b03122c16db406524c07"}
May 06 17:11:23.369922 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:23.369884    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-5b6fb99794-56z5g" podStartSLOduration=1.369869387 podStartE2EDuration="1.369869387s" podCreationTimestamp="2026-05-06 17:11:22 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-06 17:11:23.369349274 +0000 UTC m=+54.813171722" watchObservedRunningTime="2026-05-06 17:11:23.369869387 +0000 UTC m=+54.813691834"
May 06 17:11:25.361752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:25.361711    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-866jg" event={"ID":"d19c0060-6e95-4256-a8db-eeebee71e3a6","Type":"ContainerStarted","Data":"c2ee0caafa5575cc719a8b010fc2171fbac6c5ad33d5986aa953794214f7a84a"}
May 06 17:11:25.380399 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:25.380340    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-866jg" podStartSLOduration=1.201380283 podStartE2EDuration="3.380323196s" podCreationTimestamp="2026-05-06 17:11:22 +0000 UTC" firstStartedPulling="2026-05-06 17:11:22.535306156 +0000 UTC m=+53.979128597" lastFinishedPulling="2026-05-06 17:11:24.714249071 +0000 UTC m=+56.158071510" observedRunningTime="2026-05-06 17:11:25.378909793 +0000 UTC m=+56.822732243" watchObservedRunningTime="2026-05-06 17:11:25.380323196 +0000 UTC m=+56.824145642"
May 06 17:11:28.293820 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:28.293788    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-9s6rp"
May 06 17:11:31.346982 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:31.346949    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-thrt5"
May 06 17:11:33.786046 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:33.786008    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:11:33.788413 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:33.788389    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 06 17:11:33.799645 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:33.799619    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/24a44446-b589-49fd-a785-92e01601b8b4-metrics-certs\") pod \"network-metrics-daemon-cwlz9\" (UID: \"24a44446-b589-49fd-a785-92e01601b8b4\") " pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:11:33.887223 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:33.887181    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jzkfj\" (UniqueName: \"kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj\") pod \"network-check-target-jgnb4\" (UID: \"566bd2fc-87ab-4ff0-8362-cb0a84be38de\") " pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:11:33.889624 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:33.889588    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 06 17:11:33.899744 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:33.899721    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 06 17:11:33.911436 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:33.911409    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jzkfj\" (UniqueName: \"kubernetes.io/projected/566bd2fc-87ab-4ff0-8362-cb0a84be38de-kube-api-access-jzkfj\") pod \"network-check-target-jgnb4\" (UID: \"566bd2fc-87ab-4ff0-8362-cb0a84be38de\") " pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:11:34.031031 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:34.030996    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-7hgjs\""
May 06 17:11:34.039201 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:34.039122    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-cwlz9"
May 06 17:11:34.129486 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:34.129441    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-c658w\""
May 06 17:11:34.137496 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:34.137465    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:11:38.583211 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:38.583174    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-jgnb4"]
May 06 17:11:38.591202 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:11:38.591170    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod566bd2fc_87ab_4ff0_8362_cb0a84be38de.slice/crio-522c10e672795b1bf0106fc2cc593ac6bb26d14432d3b79eadc45317f3393472 WatchSource:0}: Error finding container 522c10e672795b1bf0106fc2cc593ac6bb26d14432d3b79eadc45317f3393472: Status 404 returned error can't find the container with id 522c10e672795b1bf0106fc2cc593ac6bb26d14432d3b79eadc45317f3393472
May 06 17:11:38.596041 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:38.595703    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-cwlz9"]
May 06 17:11:38.599365 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:11:38.599339    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod24a44446_b589_49fd_a785_92e01601b8b4.slice/crio-4932027fade59a9349d9e1cb4f2b8fd85616c83320062f38236ce5494a1f8a3e WatchSource:0}: Error finding container 4932027fade59a9349d9e1cb4f2b8fd85616c83320062f38236ce5494a1f8a3e: Status 404 returned error can't find the container with id 4932027fade59a9349d9e1cb4f2b8fd85616c83320062f38236ce5494a1f8a3e
May 06 17:11:39.406554 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:39.405208    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-c8c4474bc-h4vnh" event={"ID":"50600190-c8d0-4091-ad42-1655c5bb4763","Type":"ContainerStarted","Data":"d4be9372f742042e88c0afcf3bad39eaba5ef2abca01ca9fb07780c42fea2939"}
May 06 17:11:39.406554 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:39.406335    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-c8c4474bc-h4vnh"
May 06 17:11:39.410146 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:39.410116    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jgnb4" event={"ID":"566bd2fc-87ab-4ff0-8362-cb0a84be38de","Type":"ContainerStarted","Data":"522c10e672795b1bf0106fc2cc593ac6bb26d14432d3b79eadc45317f3393472"}
May 06 17:11:39.411639 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:39.411595    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cwlz9" event={"ID":"24a44446-b589-49fd-a785-92e01601b8b4","Type":"ContainerStarted","Data":"4932027fade59a9349d9e1cb4f2b8fd85616c83320062f38236ce5494a1f8a3e"}
May 06 17:11:39.425162 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:39.425118    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-c8c4474bc-h4vnh"
May 06 17:11:39.444873 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:39.444666    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-c8c4474bc-h4vnh" podStartSLOduration=1.520455826 podStartE2EDuration="17.444647781s" podCreationTimestamp="2026-05-06 17:11:22 +0000 UTC" firstStartedPulling="2026-05-06 17:11:22.597387474 +0000 UTC m=+54.041209905" lastFinishedPulling="2026-05-06 17:11:38.521579431 +0000 UTC m=+69.965401860" observedRunningTime="2026-05-06 17:11:39.425450616 +0000 UTC m=+70.869273103" watchObservedRunningTime="2026-05-06 17:11:39.444647781 +0000 UTC m=+70.888470223"
May 06 17:11:40.417098 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:40.417055    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cwlz9" event={"ID":"24a44446-b589-49fd-a785-92e01601b8b4","Type":"ContainerStarted","Data":"b0c0e2c37bcfb33c2ab98e2ae4083dc13550370cafe1949640c525cbdee9ecab"}
May 06 17:11:40.417098 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:40.417098    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-cwlz9" event={"ID":"24a44446-b589-49fd-a785-92e01601b8b4","Type":"ContainerStarted","Data":"abc1ee7ad0f49c22194d905c632837e8501fe7474e73b59d56489bad884c7e44"}
May 06 17:11:40.434707 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:40.434626    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-cwlz9" podStartSLOduration=70.021574904 podStartE2EDuration="1m11.434606411s" podCreationTimestamp="2026-05-06 17:10:29 +0000 UTC" firstStartedPulling="2026-05-06 17:11:38.6010295 +0000 UTC m=+70.044851925" lastFinishedPulling="2026-05-06 17:11:40.014060999 +0000 UTC m=+71.457883432" observedRunningTime="2026-05-06 17:11:40.434450484 +0000 UTC m=+71.878272931" watchObservedRunningTime="2026-05-06 17:11:40.434606411 +0000 UTC m=+71.878428859"
May 06 17:11:43.429022 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:43.428902    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-jgnb4" event={"ID":"566bd2fc-87ab-4ff0-8362-cb0a84be38de","Type":"ContainerStarted","Data":"ab1e9ae6dd2bd8e691b5290d0654ad08f4be61b442a09ca3b4f86db42d8f6d57"}
May 06 17:11:43.429567 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:43.429366    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:11:43.445121 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:43.445065    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-jgnb4" podStartSLOduration=70.612431594 podStartE2EDuration="1m14.445047997s" podCreationTimestamp="2026-05-06 17:10:29 +0000 UTC" firstStartedPulling="2026-05-06 17:11:38.593333653 +0000 UTC m=+70.037156077" lastFinishedPulling="2026-05-06 17:11:42.425950038 +0000 UTC m=+73.869772480" observedRunningTime="2026-05-06 17:11:43.444132237 +0000 UTC m=+74.887954683" watchObservedRunningTime="2026-05-06 17:11:43.445047997 +0000 UTC m=+74.888870445"
May 06 17:11:44.362981 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:44.362951    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-5b6fb99794-56z5g"
May 06 17:11:53.283167 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.283114    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"]
May 06 17:11:53.319546 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.319517    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"]
May 06 17:11:53.319687 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.319634    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.323067 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.323034    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 06 17:11:53.323220 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.323082    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-dockercfg-7s9p2\""
May 06 17:11:53.323220 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.323104    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 06 17:11:53.323220 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.323180    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-kube-rbac-proxy-config\""
May 06 17:11:53.323220 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.323213    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"prometheus-operator-tls\""
May 06 17:11:53.323449 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.323422    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 06 17:11:53.432844 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.432804    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cd55f\" (UniqueName: \"kubernetes.io/projected/fb3c196b-d702-4591-a31b-a48654c28dd5-kube-api-access-cd55f\") pod \"prometheus-operator-94789f4d5-f4kwf\" (UID: \"fb3c196b-d702-4591-a31b-a48654c28dd5\") " pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.433037 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.432881    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3c196b-d702-4591-a31b-a48654c28dd5-metrics-client-ca\") pod \"prometheus-operator-94789f4d5-f4kwf\" (UID: \"fb3c196b-d702-4591-a31b-a48654c28dd5\") " pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.433037 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.432924    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb3c196b-d702-4591-a31b-a48654c28dd5-prometheus-operator-tls\") pod \"prometheus-operator-94789f4d5-f4kwf\" (UID: \"fb3c196b-d702-4591-a31b-a48654c28dd5\") " pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.433037 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.432961    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb3c196b-d702-4591-a31b-a48654c28dd5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-94789f4d5-f4kwf\" (UID: \"fb3c196b-d702-4591-a31b-a48654c28dd5\") " pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.533513 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.533422    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cd55f\" (UniqueName: \"kubernetes.io/projected/fb3c196b-d702-4591-a31b-a48654c28dd5-kube-api-access-cd55f\") pod \"prometheus-operator-94789f4d5-f4kwf\" (UID: \"fb3c196b-d702-4591-a31b-a48654c28dd5\") " pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.533513 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.533511    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3c196b-d702-4591-a31b-a48654c28dd5-metrics-client-ca\") pod \"prometheus-operator-94789f4d5-f4kwf\" (UID: \"fb3c196b-d702-4591-a31b-a48654c28dd5\") " pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.533779 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.533546    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb3c196b-d702-4591-a31b-a48654c28dd5-prometheus-operator-tls\") pod \"prometheus-operator-94789f4d5-f4kwf\" (UID: \"fb3c196b-d702-4591-a31b-a48654c28dd5\") " pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.533779 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.533572    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb3c196b-d702-4591-a31b-a48654c28dd5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-94789f4d5-f4kwf\" (UID: \"fb3c196b-d702-4591-a31b-a48654c28dd5\") " pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.534138 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.534117    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/fb3c196b-d702-4591-a31b-a48654c28dd5-metrics-client-ca\") pod \"prometheus-operator-94789f4d5-f4kwf\" (UID: \"fb3c196b-d702-4591-a31b-a48654c28dd5\") " pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.536099 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.536069    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/fb3c196b-d702-4591-a31b-a48654c28dd5-prometheus-operator-kube-rbac-proxy-config\") pod \"prometheus-operator-94789f4d5-f4kwf\" (UID: \"fb3c196b-d702-4591-a31b-a48654c28dd5\") " pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.536209 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.536185    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"prometheus-operator-tls\" (UniqueName: \"kubernetes.io/secret/fb3c196b-d702-4591-a31b-a48654c28dd5-prometheus-operator-tls\") pod \"prometheus-operator-94789f4d5-f4kwf\" (UID: \"fb3c196b-d702-4591-a31b-a48654c28dd5\") " pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.541014 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.540984    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cd55f\" (UniqueName: \"kubernetes.io/projected/fb3c196b-d702-4591-a31b-a48654c28dd5-kube-api-access-cd55f\") pod \"prometheus-operator-94789f4d5-f4kwf\" (UID: \"fb3c196b-d702-4591-a31b-a48654c28dd5\") " pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.639071 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.639029    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"
May 06 17:11:53.763572 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:53.763536    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/prometheus-operator-94789f4d5-f4kwf"]
May 06 17:11:53.766427 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:11:53.766400    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb3c196b_d702_4591_a31b_a48654c28dd5.slice/crio-7a244bc6109897e4692d746f4a7975c2a03a539044d932c8de2a9ec5eb170a95 WatchSource:0}: Error finding container 7a244bc6109897e4692d746f4a7975c2a03a539044d932c8de2a9ec5eb170a95: Status 404 returned error can't find the container with id 7a244bc6109897e4692d746f4a7975c2a03a539044d932c8de2a9ec5eb170a95
May 06 17:11:54.458558 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:54.458516    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf" event={"ID":"fb3c196b-d702-4591-a31b-a48654c28dd5","Type":"ContainerStarted","Data":"7a244bc6109897e4692d746f4a7975c2a03a539044d932c8de2a9ec5eb170a95"}
May 06 17:11:56.465607 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:56.465571    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf" event={"ID":"fb3c196b-d702-4591-a31b-a48654c28dd5","Type":"ContainerStarted","Data":"9de158ca8b43dfe3f2b7923bcbb0f98ecf796217d631e9e516d997bd5a49e1fd"}
May 06 17:11:56.465607 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:56.465608    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf" event={"ID":"fb3c196b-d702-4591-a31b-a48654c28dd5","Type":"ContainerStarted","Data":"ffd532c75ad7a76ff6dba563c72ad3d45a7761d91d22aff32ba89c6f219c887d"}
May 06 17:11:56.500594 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:56.500540    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/prometheus-operator-94789f4d5-f4kwf" podStartSLOduration=1.9029903030000002 podStartE2EDuration="3.500521222s" podCreationTimestamp="2026-05-06 17:11:53 +0000 UTC" firstStartedPulling="2026-05-06 17:11:53.768332925 +0000 UTC m=+85.212155350" lastFinishedPulling="2026-05-06 17:11:55.365863841 +0000 UTC m=+86.809686269" observedRunningTime="2026-05-06 17:11:56.499327449 +0000 UTC m=+87.943149933" watchObservedRunningTime="2026-05-06 17:11:56.500521222 +0000 UTC m=+87.944343671"
May 06 17:11:58.705295 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.705253    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"]
May 06 17:11:58.730852 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.730826    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-msrpm"]
May 06 17:11:58.731043 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.731022    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.733637 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.733614    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
May 06 17:11:58.733811 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.733791    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
May 06 17:11:58.733901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.733792    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
May 06 17:11:58.733901 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.733865    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-n9dsq\""
May 06 17:11:58.752330 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.752310    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"]
May 06 17:11:58.752448 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.752423    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.754509 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.754490    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 06 17:11:58.754663 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.754537    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 06 17:11:58.754868 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.754846    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 06 17:11:58.754991 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.754589    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-7k9wf\""
May 06 17:11:58.770639 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.770620    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-metrics-client-ca\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.770761 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.770666    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-volume-directive-shadow\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.770761 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.770693    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.770761 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.770730    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hc4wg\" (UniqueName: \"kubernetes.io/projected/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-api-access-hc4wg\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.770920 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.770805    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-state-metrics-tls\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.770920 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.770852    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.872055 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872023    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-state-metrics-tls\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.872055 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872061    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-textfile\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.872313 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872086    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.872313 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872113    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.872313 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872147    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-accelerators-collector-config\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.872313 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:58.872184    2560 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
May 06 17:11:58.872313 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:11:58.872255    2560 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-state-metrics-tls podName:f99cc8a8-6961-4cd2-b612-920c74b2b4a3 nodeName:}" failed. No retries permitted until 2026-05-06 17:11:59.372234906 +0000 UTC m=+90.816057337 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-state-metrics-tls") pod "kube-state-metrics-7764dcf94f-rkpbs" (UID: "f99cc8a8-6961-4cd2-b612-920c74b2b4a3") : secret "kube-state-metrics-tls" not found
May 06 17:11:58.872313 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872188    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-sys\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.872313 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872303    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-metrics-client-ca\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.872665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872340    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-metrics-client-ca\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.872665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872372    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nqrqz\" (UniqueName: \"kubernetes.io/projected/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-kube-api-access-nqrqz\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.872665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872417    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-tls\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.872665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872454    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-volume-directive-shadow\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.872665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872483    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-wtmp\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.872665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872514    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.872665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872580    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-root\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.872665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872618    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hc4wg\" (UniqueName: \"kubernetes.io/projected/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-api-access-hc4wg\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.873049 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.872888    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-volume-directive-shadow\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.873099 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.873068    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-metrics-client-ca\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.873175 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.873120    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.874875 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.874851    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.883521 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.883498    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hc4wg\" (UniqueName: \"kubernetes.io/projected/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-api-access-hc4wg\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:58.973206 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973104    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-tls\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.973206 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973142    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-wtmp\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.973427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973261    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-root\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.973427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973284    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-wtmp\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.973427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973327    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-textfile\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.973427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973349    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.973427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973376    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-accelerators-collector-config\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.973427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973407    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-sys\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.973790 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973370    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-root\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.973790 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973435    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-metrics-client-ca\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.973790 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973509    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nqrqz\" (UniqueName: \"kubernetes.io/projected/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-kube-api-access-nqrqz\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.973790 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973511    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-sys\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.973790 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973683    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-textfile\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.974041 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.973969    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-metrics-client-ca\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.974106 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.974078    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-accelerators-collector-config\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.975592 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.975568    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.975706 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.975687    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-node-exporter-tls\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:58.981628 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:58.981606    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nqrqz\" (UniqueName: \"kubernetes.io/projected/e5bf6bb6-c650-4f3a-996d-0cfe4d87178a-kube-api-access-nqrqz\") pod \"node-exporter-msrpm\" (UID: \"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a\") " pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:59.061165 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:59.061130    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-msrpm"
May 06 17:11:59.068878 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:11:59.068842    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode5bf6bb6_c650_4f3a_996d_0cfe4d87178a.slice/crio-5d975dcb38b112171b695e9159adc707fe9ef565a50e7894232254bdc8d1c5cb WatchSource:0}: Error finding container 5d975dcb38b112171b695e9159adc707fe9ef565a50e7894232254bdc8d1c5cb: Status 404 returned error can't find the container with id 5d975dcb38b112171b695e9159adc707fe9ef565a50e7894232254bdc8d1c5cb
May 06 17:11:59.377450 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:59.377371    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-state-metrics-tls\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:59.379620 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:59.379594    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/f99cc8a8-6961-4cd2-b612-920c74b2b4a3-kube-state-metrics-tls\") pod \"kube-state-metrics-7764dcf94f-rkpbs\" (UID: \"f99cc8a8-6961-4cd2-b612-920c74b2b4a3\") " pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:59.476868 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:59.476823    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-msrpm" event={"ID":"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a","Type":"ContainerStarted","Data":"5d975dcb38b112171b695e9159adc707fe9ef565a50e7894232254bdc8d1c5cb"}
May 06 17:11:59.640568 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:59.640485    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"
May 06 17:11:59.797590 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:11:59.797559    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs"]
May 06 17:11:59.801418 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:11:59.801385    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf99cc8a8_6961_4cd2_b612_920c74b2b4a3.slice/crio-b6d6bd3e287b7004c18e176fd13a968ad2b4381cd1db06604e1fa7b317c200b6 WatchSource:0}: Error finding container b6d6bd3e287b7004c18e176fd13a968ad2b4381cd1db06604e1fa7b317c200b6: Status 404 returned error can't find the container with id b6d6bd3e287b7004c18e176fd13a968ad2b4381cd1db06604e1fa7b317c200b6
May 06 17:12:00.482836 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.482796    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs" event={"ID":"f99cc8a8-6961-4cd2-b612-920c74b2b4a3","Type":"ContainerStarted","Data":"b6d6bd3e287b7004c18e176fd13a968ad2b4381cd1db06604e1fa7b317c200b6"}
May 06 17:12:00.720428 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.720390    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-6d65767784-9t7ms"]
May 06 17:12:00.722970 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.722947    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.725485 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.725453    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
May 06 17:12:00.725627 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.725517    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-d2dvq\""
May 06 17:12:00.725627 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.725564    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
May 06 17:12:00.725627 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.725455    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
May 06 17:12:00.725627 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.725523    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
May 06 17:12:00.725827 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.725469    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
May 06 17:12:00.725927 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.725907    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-bpa4jb8tbjtdm\""
May 06 17:12:00.739940 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.739873    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d65767784-9t7ms"]
May 06 17:12:00.788923 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.788888    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-tls\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.788923 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.788923    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-grpc-tls\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.789126 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.788941    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-metrics-client-ca\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.789126 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.788966    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.789126 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.788991    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.789126 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.789084    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.789126 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.789118    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.789301 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.789209    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sk8xt\" (UniqueName: \"kubernetes.io/projected/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-kube-api-access-sk8xt\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.889714 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.889673    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.890206 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.889725    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.890206 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.889775    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.890206 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.889803    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.890206 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.889870    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sk8xt\" (UniqueName: \"kubernetes.io/projected/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-kube-api-access-sk8xt\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.890206 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.889901    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-tls\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.890206 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.889928    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-grpc-tls\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.890206 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.889956    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-metrics-client-ca\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.890741 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.890716    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-metrics-client-ca\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.892921 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.892894    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.893138 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.893090    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.893521 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.893495    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.893589 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.893535    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-grpc-tls\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.893589 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.893537    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.893703 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.893604    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-secret-thanos-querier-tls\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:00.901244 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:00.901215    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sk8xt\" (UniqueName: \"kubernetes.io/projected/c32ccae0-4d99-4a68-b85a-2406d8b6f16a-kube-api-access-sk8xt\") pod \"thanos-querier-6d65767784-9t7ms\" (UID: \"c32ccae0-4d99-4a68-b85a-2406d8b6f16a\") " pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:01.034763 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:01.034680    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:01.456704 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:01.456678    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-6d65767784-9t7ms"]
May 06 17:12:01.459303 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:12:01.459272    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc32ccae0_4d99_4a68_b85a_2406d8b6f16a.slice/crio-b53fac4f7a75dab3ffd2a37de96dea754f2ab89e71153338923f29f7d19644a7 WatchSource:0}: Error finding container b53fac4f7a75dab3ffd2a37de96dea754f2ab89e71153338923f29f7d19644a7: Status 404 returned error can't find the container with id b53fac4f7a75dab3ffd2a37de96dea754f2ab89e71153338923f29f7d19644a7
May 06 17:12:01.488402 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:01.488332    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs" event={"ID":"f99cc8a8-6961-4cd2-b612-920c74b2b4a3","Type":"ContainerStarted","Data":"4b3c1fa8bb3344506dea4d12b94c71d44df25caed885a42e1b767956a80b4ea5"}
May 06 17:12:01.488402 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:01.488379    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs" event={"ID":"f99cc8a8-6961-4cd2-b612-920c74b2b4a3","Type":"ContainerStarted","Data":"1438a3e51831243fa023d23652849d5a8ca5cd1473a559d31fc034fddacacd05"}
May 06 17:12:01.490536 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:01.490372    2560 generic.go:358] "Generic (PLEG): container finished" podID="e5bf6bb6-c650-4f3a-996d-0cfe4d87178a" containerID="562195e58a8e9bd9e94ffc46c052635a9052c19ce9e0af27b3dda0d9a9e2e67f" exitCode=0
May 06 17:12:01.490536 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:01.490498    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-msrpm" event={"ID":"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a","Type":"ContainerDied","Data":"562195e58a8e9bd9e94ffc46c052635a9052c19ce9e0af27b3dda0d9a9e2e67f"}
May 06 17:12:01.494952 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:01.494850    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms" event={"ID":"c32ccae0-4d99-4a68-b85a-2406d8b6f16a","Type":"ContainerStarted","Data":"b53fac4f7a75dab3ffd2a37de96dea754f2ab89e71153338923f29f7d19644a7"}
May 06 17:12:02.500105 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:02.500052    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs" event={"ID":"f99cc8a8-6961-4cd2-b612-920c74b2b4a3","Type":"ContainerStarted","Data":"bdcb1dbe7c53d6daa8a61cae93ee5c6f31372a14ef68ae68edaa8f2ff1ee9d77"}
May 06 17:12:02.502429 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:02.502397    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-msrpm" event={"ID":"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a","Type":"ContainerStarted","Data":"c1228492487ea66abeb692228f13e61970b658bfb83050322bae8e3d34b3cb3b"}
May 06 17:12:02.502570 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:02.502432    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-msrpm" event={"ID":"e5bf6bb6-c650-4f3a-996d-0cfe4d87178a","Type":"ContainerStarted","Data":"b8cd769a6f3a071b9f394a8378917c9bb38297dc58d00075de0196daf2fee5f8"}
May 06 17:12:02.520411 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:02.519859    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-7764dcf94f-rkpbs" podStartSLOduration=2.99807078 podStartE2EDuration="4.519839294s" podCreationTimestamp="2026-05-06 17:11:58 +0000 UTC" firstStartedPulling="2026-05-06 17:11:59.80363431 +0000 UTC m=+91.247456746" lastFinishedPulling="2026-05-06 17:12:01.325402811 +0000 UTC m=+92.769225260" observedRunningTime="2026-05-06 17:12:02.519064667 +0000 UTC m=+93.962887116" watchObservedRunningTime="2026-05-06 17:12:02.519839294 +0000 UTC m=+93.963661749"
May 06 17:12:02.538801 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:02.538743    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-msrpm" podStartSLOduration=3.118600684 podStartE2EDuration="4.538726902s" podCreationTimestamp="2026-05-06 17:11:58 +0000 UTC" firstStartedPulling="2026-05-06 17:11:59.070718447 +0000 UTC m=+90.514540873" lastFinishedPulling="2026-05-06 17:12:00.490844661 +0000 UTC m=+91.934667091" observedRunningTime="2026-05-06 17:12:02.536487485 +0000 UTC m=+93.980309931" watchObservedRunningTime="2026-05-06 17:12:02.538726902 +0000 UTC m=+93.982549352"
May 06 17:12:04.510093 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:04.510055    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms" event={"ID":"c32ccae0-4d99-4a68-b85a-2406d8b6f16a","Type":"ContainerStarted","Data":"f6d46f29639950c44137fc56fa828e3a798867077b06da3c99b32674b1817df3"}
May 06 17:12:04.510093 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:04.510093    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms" event={"ID":"c32ccae0-4d99-4a68-b85a-2406d8b6f16a","Type":"ContainerStarted","Data":"18f8868c5f3180f1e2adcf6906d9092674a840809b305544edb1e99201e7e081"}
May 06 17:12:04.510526 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:04.510105    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms" event={"ID":"c32ccae0-4d99-4a68-b85a-2406d8b6f16a","Type":"ContainerStarted","Data":"857576fade32563eb5d786caf4758c37c7af568f2461f67ac7a93b4d8e065c9b"}
May 06 17:12:06.518602 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:06.518519    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms" event={"ID":"c32ccae0-4d99-4a68-b85a-2406d8b6f16a","Type":"ContainerStarted","Data":"b8b8cab35196ac0fa0df386aa4082f6e57c1e9d11907513bd72128d499571efb"}
May 06 17:12:06.518602 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:06.518556    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms" event={"ID":"c32ccae0-4d99-4a68-b85a-2406d8b6f16a","Type":"ContainerStarted","Data":"057b095d2739fd91b6686e7943eed4473642a98dcadf999393b82cf420b0b461"}
May 06 17:12:06.518602 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:06.518566    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms" event={"ID":"c32ccae0-4d99-4a68-b85a-2406d8b6f16a","Type":"ContainerStarted","Data":"485e473084079d6c2fda9a20f4a02f534173a0e1f14cc88c85a87634659e7469"}
May 06 17:12:06.519022 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:06.518677    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:06.542792 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:06.542734    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms" podStartSLOduration=2.365928404 podStartE2EDuration="6.542701327s" podCreationTimestamp="2026-05-06 17:12:00 +0000 UTC" firstStartedPulling="2026-05-06 17:12:01.461520755 +0000 UTC m=+92.905343181" lastFinishedPulling="2026-05-06 17:12:05.638293674 +0000 UTC m=+97.082116104" observedRunningTime="2026-05-06 17:12:06.541228005 +0000 UTC m=+97.985050453" watchObservedRunningTime="2026-05-06 17:12:06.542701327 +0000 UTC m=+97.986523766"
May 06 17:12:12.527137 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:12.527107    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-6d65767784-9t7ms"
May 06 17:12:14.435300 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:12:14.435268    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-jgnb4"
May 06 17:13:56.778209 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:56.778175    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-6jh2w"]
May 06 17:13:56.780111 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:56.780095    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jh2w"
May 06 17:13:56.782439 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:56.782416    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 06 17:13:56.787767 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:56.787746    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6jh2w"]
May 06 17:13:56.950354 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:56.950313    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9-kubelet-config\") pod \"global-pull-secret-syncer-6jh2w\" (UID: \"3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9\") " pod="kube-system/global-pull-secret-syncer-6jh2w"
May 06 17:13:56.950534 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:56.950375    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9-dbus\") pod \"global-pull-secret-syncer-6jh2w\" (UID: \"3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9\") " pod="kube-system/global-pull-secret-syncer-6jh2w"
May 06 17:13:56.950534 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:56.950410    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9-original-pull-secret\") pod \"global-pull-secret-syncer-6jh2w\" (UID: \"3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9\") " pod="kube-system/global-pull-secret-syncer-6jh2w"
May 06 17:13:57.051813 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:57.051714    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9-dbus\") pod \"global-pull-secret-syncer-6jh2w\" (UID: \"3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9\") " pod="kube-system/global-pull-secret-syncer-6jh2w"
May 06 17:13:57.051813 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:57.051765    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9-original-pull-secret\") pod \"global-pull-secret-syncer-6jh2w\" (UID: \"3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9\") " pod="kube-system/global-pull-secret-syncer-6jh2w"
May 06 17:13:57.051813 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:57.051803    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9-kubelet-config\") pod \"global-pull-secret-syncer-6jh2w\" (UID: \"3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9\") " pod="kube-system/global-pull-secret-syncer-6jh2w"
May 06 17:13:57.052051 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:57.051877    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9-kubelet-config\") pod \"global-pull-secret-syncer-6jh2w\" (UID: \"3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9\") " pod="kube-system/global-pull-secret-syncer-6jh2w"
May 06 17:13:57.052051 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:57.051942    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9-dbus\") pod \"global-pull-secret-syncer-6jh2w\" (UID: \"3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9\") " pod="kube-system/global-pull-secret-syncer-6jh2w"
May 06 17:13:57.054110 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:57.054087    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9-original-pull-secret\") pod \"global-pull-secret-syncer-6jh2w\" (UID: \"3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9\") " pod="kube-system/global-pull-secret-syncer-6jh2w"
May 06 17:13:57.089200 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:57.089146    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6jh2w"
May 06 17:13:57.209926 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:57.209894    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6jh2w"]
May 06 17:13:57.212882 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:13:57.212849    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3c0cf9e2_df1e_4740_89a9_5a9830ceb6d9.slice/crio-a4476b7cd1ad99b1a2d03655e14ec2e699953ff0ead2a22b2e9599730c261ac3 WatchSource:0}: Error finding container a4476b7cd1ad99b1a2d03655e14ec2e699953ff0ead2a22b2e9599730c261ac3: Status 404 returned error can't find the container with id a4476b7cd1ad99b1a2d03655e14ec2e699953ff0ead2a22b2e9599730c261ac3
May 06 17:13:57.804496 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:13:57.804455    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6jh2w" event={"ID":"3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9","Type":"ContainerStarted","Data":"a4476b7cd1ad99b1a2d03655e14ec2e699953ff0ead2a22b2e9599730c261ac3"}
May 06 17:14:01.821017 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:01.820985    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6jh2w" event={"ID":"3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9","Type":"ContainerStarted","Data":"49bd20aeef9e756b8fac3161c56a0b8003b8e0c7aac3adc74d4967cf9d0afd13"}
May 06 17:14:01.836869 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:01.836811    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6jh2w" podStartSLOduration=2.044154271 podStartE2EDuration="5.836797597s" podCreationTimestamp="2026-05-06 17:13:56 +0000 UTC" firstStartedPulling="2026-05-06 17:13:57.214488569 +0000 UTC m=+208.658310994" lastFinishedPulling="2026-05-06 17:14:01.00713188 +0000 UTC m=+212.450954320" observedRunningTime="2026-05-06 17:14:01.836255387 +0000 UTC m=+213.280077834" watchObservedRunningTime="2026-05-06 17:14:01.836797597 +0000 UTC m=+213.280620044"
May 06 17:14:31.127786 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.127751    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"]
May 06 17:14:31.131377 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.131350    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:31.133670 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.133646    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 06 17:14:31.134371 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.134357    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tpsdd\""
May 06 17:14:31.134495 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.134470    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 06 17:14:31.139336 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.139315    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"]
May 06 17:14:31.199552 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.199512    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5vkn8\" (UniqueName: \"kubernetes.io/projected/3f0ecc75-64be-4008-be39-5c700753e446-kube-api-access-5vkn8\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc\" (UID: \"3f0ecc75-64be-4008-be39-5c700753e446\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:31.199552 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.199556    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f0ecc75-64be-4008-be39-5c700753e446-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc\" (UID: \"3f0ecc75-64be-4008-be39-5c700753e446\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:31.199763 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.199582    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f0ecc75-64be-4008-be39-5c700753e446-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc\" (UID: \"3f0ecc75-64be-4008-be39-5c700753e446\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:31.300253 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.300219    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5vkn8\" (UniqueName: \"kubernetes.io/projected/3f0ecc75-64be-4008-be39-5c700753e446-kube-api-access-5vkn8\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc\" (UID: \"3f0ecc75-64be-4008-be39-5c700753e446\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:31.300334 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.300262    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f0ecc75-64be-4008-be39-5c700753e446-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc\" (UID: \"3f0ecc75-64be-4008-be39-5c700753e446\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:31.300334 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.300288    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f0ecc75-64be-4008-be39-5c700753e446-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc\" (UID: \"3f0ecc75-64be-4008-be39-5c700753e446\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:31.300631 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.300607    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f0ecc75-64be-4008-be39-5c700753e446-util\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc\" (UID: \"3f0ecc75-64be-4008-be39-5c700753e446\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:31.300672 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.300637    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f0ecc75-64be-4008-be39-5c700753e446-bundle\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc\" (UID: \"3f0ecc75-64be-4008-be39-5c700753e446\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:31.308695 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.308662    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5vkn8\" (UniqueName: \"kubernetes.io/projected/3f0ecc75-64be-4008-be39-5c700753e446-kube-api-access-5vkn8\") pod \"2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc\" (UID: \"3f0ecc75-64be-4008-be39-5c700753e446\") " pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:31.441593 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.441564    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:31.566049 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.566017    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"]
May 06 17:14:31.570505 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:14:31.570479    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3f0ecc75_64be_4008_be39_5c700753e446.slice/crio-cfe6609217e618de6a8a1c9210779b64e77ebf226e6a23df6d129232d8eee20a WatchSource:0}: Error finding container cfe6609217e618de6a8a1c9210779b64e77ebf226e6a23df6d129232d8eee20a: Status 404 returned error can't find the container with id cfe6609217e618de6a8a1c9210779b64e77ebf226e6a23df6d129232d8eee20a
May 06 17:14:31.902838 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:31.902747    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc" event={"ID":"3f0ecc75-64be-4008-be39-5c700753e446","Type":"ContainerStarted","Data":"cfe6609217e618de6a8a1c9210779b64e77ebf226e6a23df6d129232d8eee20a"}
May 06 17:14:37.920452 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:37.920422    2560 generic.go:358] "Generic (PLEG): container finished" podID="3f0ecc75-64be-4008-be39-5c700753e446" containerID="88dcaa45a37e515327af4f7ce60928371760ec590fa37d529915366e8b5a9280" exitCode=0
May 06 17:14:37.920810 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:37.920508    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc" event={"ID":"3f0ecc75-64be-4008-be39-5c700753e446","Type":"ContainerDied","Data":"88dcaa45a37e515327af4f7ce60928371760ec590fa37d529915366e8b5a9280"}
May 06 17:14:40.930570 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:40.930525    2560 generic.go:358] "Generic (PLEG): container finished" podID="3f0ecc75-64be-4008-be39-5c700753e446" containerID="78ededfeeb1db3eefce84566100a680ef9ef2da06dff0f027c7d73ee3e2f7b76" exitCode=0
May 06 17:14:40.930935 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:40.930592    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc" event={"ID":"3f0ecc75-64be-4008-be39-5c700753e446","Type":"ContainerDied","Data":"78ededfeeb1db3eefce84566100a680ef9ef2da06dff0f027c7d73ee3e2f7b76"}
May 06 17:14:47.953634 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:47.953599    2560 generic.go:358] "Generic (PLEG): container finished" podID="3f0ecc75-64be-4008-be39-5c700753e446" containerID="ddeab7543a7670ed4b04f0ab8def1c695d57e2ddf6b2acb0f34a61c70b84efd5" exitCode=0
May 06 17:14:47.953993 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:47.953670    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc" event={"ID":"3f0ecc75-64be-4008-be39-5c700753e446","Type":"ContainerDied","Data":"ddeab7543a7670ed4b04f0ab8def1c695d57e2ddf6b2acb0f34a61c70b84efd5"}
May 06 17:14:49.076105 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.076079    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:49.142804 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.142773    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f0ecc75-64be-4008-be39-5c700753e446-util\") pod \"3f0ecc75-64be-4008-be39-5c700753e446\" (UID: \"3f0ecc75-64be-4008-be39-5c700753e446\") "
May 06 17:14:49.142945 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.142838    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5vkn8\" (UniqueName: \"kubernetes.io/projected/3f0ecc75-64be-4008-be39-5c700753e446-kube-api-access-5vkn8\") pod \"3f0ecc75-64be-4008-be39-5c700753e446\" (UID: \"3f0ecc75-64be-4008-be39-5c700753e446\") "
May 06 17:14:49.142945 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.142854    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f0ecc75-64be-4008-be39-5c700753e446-bundle\") pod \"3f0ecc75-64be-4008-be39-5c700753e446\" (UID: \"3f0ecc75-64be-4008-be39-5c700753e446\") "
May 06 17:14:49.143531 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.143506    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f0ecc75-64be-4008-be39-5c700753e446-bundle" (OuterVolumeSpecName: "bundle") pod "3f0ecc75-64be-4008-be39-5c700753e446" (UID: "3f0ecc75-64be-4008-be39-5c700753e446"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:14:49.144980 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.144962    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3f0ecc75-64be-4008-be39-5c700753e446-kube-api-access-5vkn8" (OuterVolumeSpecName: "kube-api-access-5vkn8") pod "3f0ecc75-64be-4008-be39-5c700753e446" (UID: "3f0ecc75-64be-4008-be39-5c700753e446"). InnerVolumeSpecName "kube-api-access-5vkn8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:14:49.146643 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.146622    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/3f0ecc75-64be-4008-be39-5c700753e446-util" (OuterVolumeSpecName: "util") pod "3f0ecc75-64be-4008-be39-5c700753e446" (UID: "3f0ecc75-64be-4008-be39-5c700753e446"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:14:49.244146 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.244061    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5vkn8\" (UniqueName: \"kubernetes.io/projected/3f0ecc75-64be-4008-be39-5c700753e446-kube-api-access-5vkn8\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:14:49.244146 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.244101    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/3f0ecc75-64be-4008-be39-5c700753e446-bundle\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:14:49.244146 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.244111    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/3f0ecc75-64be-4008-be39-5c700753e446-util\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:14:49.961312 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.961278    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc" event={"ID":"3f0ecc75-64be-4008-be39-5c700753e446","Type":"ContainerDied","Data":"cfe6609217e618de6a8a1c9210779b64e77ebf226e6a23df6d129232d8eee20a"}
May 06 17:14:49.961312 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.961314    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="cfe6609217e618de6a8a1c9210779b64e77ebf226e6a23df6d129232d8eee20a"
May 06 17:14:49.961511 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:49.961333    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/2bb52b92bd31ddf2ebbc335370ac517be50e61a93c2fe375393413c19d5lcwc"
May 06 17:14:54.104094 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.104062    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv"]
May 06 17:14:54.104517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.104409    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f0ecc75-64be-4008-be39-5c700753e446" containerName="util"
May 06 17:14:54.104517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.104424    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f0ecc75-64be-4008-be39-5c700753e446" containerName="util"
May 06 17:14:54.104517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.104432    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f0ecc75-64be-4008-be39-5c700753e446" containerName="pull"
May 06 17:14:54.104517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.104438    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f0ecc75-64be-4008-be39-5c700753e446" containerName="pull"
May 06 17:14:54.104517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.104452    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3f0ecc75-64be-4008-be39-5c700753e446" containerName="extract"
May 06 17:14:54.104517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.104457    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3f0ecc75-64be-4008-be39-5c700753e446" containerName="extract"
May 06 17:14:54.104517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.104503    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="3f0ecc75-64be-4008-be39-5c700753e446" containerName="extract"
May 06 17:14:54.151303 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.151276    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv"]
May 06 17:14:54.151470 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.151424    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv"
May 06 17:14:54.154207 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.154179    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"openshift-service-ca.crt\""
May 06 17:14:54.154207 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.154194    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager-operator\"/\"cert-manager-operator-controller-manager-dockercfg-mzm4h\""
May 06 17:14:54.154383 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.154221    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager-operator\"/\"kube-root-ca.crt\""
May 06 17:14:54.280269 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.280229    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f98863b-be83-453c-a57d-a21e1a9bc753-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-4f6sv\" (UID: \"8f98863b-be83-453c-a57d-a21e1a9bc753\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv"
May 06 17:14:54.280450 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.280279    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dgqfz\" (UniqueName: \"kubernetes.io/projected/8f98863b-be83-453c-a57d-a21e1a9bc753-kube-api-access-dgqfz\") pod \"cert-manager-operator-controller-manager-54b9655956-4f6sv\" (UID: \"8f98863b-be83-453c-a57d-a21e1a9bc753\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv"
May 06 17:14:54.380978 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.380901    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f98863b-be83-453c-a57d-a21e1a9bc753-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-4f6sv\" (UID: \"8f98863b-be83-453c-a57d-a21e1a9bc753\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv"
May 06 17:14:54.380978 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.380941    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dgqfz\" (UniqueName: \"kubernetes.io/projected/8f98863b-be83-453c-a57d-a21e1a9bc753-kube-api-access-dgqfz\") pod \"cert-manager-operator-controller-manager-54b9655956-4f6sv\" (UID: \"8f98863b-be83-453c-a57d-a21e1a9bc753\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv"
May 06 17:14:54.381307 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.381285    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/8f98863b-be83-453c-a57d-a21e1a9bc753-tmp\") pod \"cert-manager-operator-controller-manager-54b9655956-4f6sv\" (UID: \"8f98863b-be83-453c-a57d-a21e1a9bc753\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv"
May 06 17:14:54.389421 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.389389    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dgqfz\" (UniqueName: \"kubernetes.io/projected/8f98863b-be83-453c-a57d-a21e1a9bc753-kube-api-access-dgqfz\") pod \"cert-manager-operator-controller-manager-54b9655956-4f6sv\" (UID: \"8f98863b-be83-453c-a57d-a21e1a9bc753\") " pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv"
May 06 17:14:54.460683 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.460650    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv"
May 06 17:14:54.586113 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.586085    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv"]
May 06 17:14:54.589488 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:14:54.589458    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8f98863b_be83_453c_a57d_a21e1a9bc753.slice/crio-8e2ae4db29a57680284eade17f9090a750d237e1517444af170f199ef06f1db4 WatchSource:0}: Error finding container 8e2ae4db29a57680284eade17f9090a750d237e1517444af170f199ef06f1db4: Status 404 returned error can't find the container with id 8e2ae4db29a57680284eade17f9090a750d237e1517444af170f199ef06f1db4
May 06 17:14:54.977820 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:54.977780    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv" event={"ID":"8f98863b-be83-453c-a57d-a21e1a9bc753","Type":"ContainerStarted","Data":"8e2ae4db29a57680284eade17f9090a750d237e1517444af170f199ef06f1db4"}
May 06 17:14:56.986139 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:56.986099    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv" event={"ID":"8f98863b-be83-453c-a57d-a21e1a9bc753","Type":"ContainerStarted","Data":"39bdf289da5b75f94cba292d66e2eca6b947e3b23bebfeb421aecf3f0e4717ff"}
May 06 17:14:57.006497 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:57.006442    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager-operator/cert-manager-operator-controller-manager-54b9655956-4f6sv" podStartSLOduration=0.736035039 podStartE2EDuration="3.006425859s" podCreationTimestamp="2026-05-06 17:14:54 +0000 UTC" firstStartedPulling="2026-05-06 17:14:54.59183881 +0000 UTC m=+266.035661239" lastFinishedPulling="2026-05-06 17:14:56.862229634 +0000 UTC m=+268.306052059" observedRunningTime="2026-05-06 17:14:57.004522418 +0000 UTC m=+268.448344905" watchObservedRunningTime="2026-05-06 17:14:57.006425859 +0000 UTC m=+268.450248305"
May 06 17:14:57.968964 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:57.968924    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"]
May 06 17:14:57.990112 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:57.990077    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"]
May 06 17:14:57.990516 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:57.990351    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:14:57.993033 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:57.993008    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 06 17:14:57.993184 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:57.993096    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 06 17:14:57.993665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:57.993646    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tpsdd\""
May 06 17:14:58.111478 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.111436    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edccf5cc-c860-49e3-a67e-14cc757eb66e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q\" (UID: \"edccf5cc-c860-49e3-a67e-14cc757eb66e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:14:58.111478 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.111495    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gd64w\" (UniqueName: \"kubernetes.io/projected/edccf5cc-c860-49e3-a67e-14cc757eb66e-kube-api-access-gd64w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q\" (UID: \"edccf5cc-c860-49e3-a67e-14cc757eb66e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:14:58.111713 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.111606    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edccf5cc-c860-49e3-a67e-14cc757eb66e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q\" (UID: \"edccf5cc-c860-49e3-a67e-14cc757eb66e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:14:58.212931 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.212888    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gd64w\" (UniqueName: \"kubernetes.io/projected/edccf5cc-c860-49e3-a67e-14cc757eb66e-kube-api-access-gd64w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q\" (UID: \"edccf5cc-c860-49e3-a67e-14cc757eb66e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:14:58.213093 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.212966    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edccf5cc-c860-49e3-a67e-14cc757eb66e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q\" (UID: \"edccf5cc-c860-49e3-a67e-14cc757eb66e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:14:58.213093 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.213006    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edccf5cc-c860-49e3-a67e-14cc757eb66e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q\" (UID: \"edccf5cc-c860-49e3-a67e-14cc757eb66e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:14:58.213388 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.213367    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edccf5cc-c860-49e3-a67e-14cc757eb66e-util\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q\" (UID: \"edccf5cc-c860-49e3-a67e-14cc757eb66e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:14:58.213452 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.213435    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edccf5cc-c860-49e3-a67e-14cc757eb66e-bundle\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q\" (UID: \"edccf5cc-c860-49e3-a67e-14cc757eb66e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:14:58.226128 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.226060    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gd64w\" (UniqueName: \"kubernetes.io/projected/edccf5cc-c860-49e3-a67e-14cc757eb66e-kube-api-access-gd64w\") pod \"77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q\" (UID: \"edccf5cc-c860-49e3-a67e-14cc757eb66e\") " pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:14:58.299755 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.299723    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:14:58.428215 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.428177    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"]
May 06 17:14:58.431001 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:14:58.430969    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podedccf5cc_c860_49e3_a67e_14cc757eb66e.slice/crio-da7e842a862b008934f8e893bd6e68e3b469b9ec0b4f777c7e5846ecf9643a04 WatchSource:0}: Error finding container da7e842a862b008934f8e893bd6e68e3b469b9ec0b4f777c7e5846ecf9643a04: Status 404 returned error can't find the container with id da7e842a862b008934f8e893bd6e68e3b469b9ec0b4f777c7e5846ecf9643a04
May 06 17:14:58.992815 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.992780    2560 generic.go:358] "Generic (PLEG): container finished" podID="edccf5cc-c860-49e3-a67e-14cc757eb66e" containerID="c58e5693fb809481313295053591006193e0a4ad9a7075beed3214102d59c728" exitCode=0
May 06 17:14:58.993255 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.992866    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q" event={"ID":"edccf5cc-c860-49e3-a67e-14cc757eb66e","Type":"ContainerDied","Data":"c58e5693fb809481313295053591006193e0a4ad9a7075beed3214102d59c728"}
May 06 17:14:58.993255 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:14:58.992903    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q" event={"ID":"edccf5cc-c860-49e3-a67e-14cc757eb66e","Type":"ContainerStarted","Data":"da7e842a862b008934f8e893bd6e68e3b469b9ec0b4f777c7e5846ecf9643a04"}
May 06 17:15:01.257271 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.257237    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-dvfp5"]
May 06 17:15:01.268335 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.268303    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-dvfp5"
May 06 17:15:01.270791 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.270768    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"openshift-service-ca.crt\""
May 06 17:15:01.271043 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.271020    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-dvfp5"]
May 06 17:15:01.271472 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.271450    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"cert-manager\"/\"kube-root-ca.crt\""
May 06 17:15:01.271666 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.271644    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-cainjector-dockercfg-2b2hw\""
May 06 17:15:01.339401 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.339352    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34ccf2cf-33e4-4856-a8c6-a2944f558cd8-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-dvfp5\" (UID: \"34ccf2cf-33e4-4856-a8c6-a2944f558cd8\") " pod="cert-manager/cert-manager-cainjector-68b757865b-dvfp5"
May 06 17:15:01.339594 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.339461    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t8rhp\" (UniqueName: \"kubernetes.io/projected/34ccf2cf-33e4-4856-a8c6-a2944f558cd8-kube-api-access-t8rhp\") pod \"cert-manager-cainjector-68b757865b-dvfp5\" (UID: \"34ccf2cf-33e4-4856-a8c6-a2944f558cd8\") " pod="cert-manager/cert-manager-cainjector-68b757865b-dvfp5"
May 06 17:15:01.440505 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.440318    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t8rhp\" (UniqueName: \"kubernetes.io/projected/34ccf2cf-33e4-4856-a8c6-a2944f558cd8-kube-api-access-t8rhp\") pod \"cert-manager-cainjector-68b757865b-dvfp5\" (UID: \"34ccf2cf-33e4-4856-a8c6-a2944f558cd8\") " pod="cert-manager/cert-manager-cainjector-68b757865b-dvfp5"
May 06 17:15:01.440505 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.440420    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34ccf2cf-33e4-4856-a8c6-a2944f558cd8-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-dvfp5\" (UID: \"34ccf2cf-33e4-4856-a8c6-a2944f558cd8\") " pod="cert-manager/cert-manager-cainjector-68b757865b-dvfp5"
May 06 17:15:01.448745 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.448707    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/34ccf2cf-33e4-4856-a8c6-a2944f558cd8-bound-sa-token\") pod \"cert-manager-cainjector-68b757865b-dvfp5\" (UID: \"34ccf2cf-33e4-4856-a8c6-a2944f558cd8\") " pod="cert-manager/cert-manager-cainjector-68b757865b-dvfp5"
May 06 17:15:01.449331 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.449307    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t8rhp\" (UniqueName: \"kubernetes.io/projected/34ccf2cf-33e4-4856-a8c6-a2944f558cd8-kube-api-access-t8rhp\") pod \"cert-manager-cainjector-68b757865b-dvfp5\" (UID: \"34ccf2cf-33e4-4856-a8c6-a2944f558cd8\") " pod="cert-manager/cert-manager-cainjector-68b757865b-dvfp5"
May 06 17:15:01.590862 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.590832    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-cainjector-68b757865b-dvfp5"
May 06 17:15:01.724282 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:01.724248    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-cainjector-68b757865b-dvfp5"]
May 06 17:15:01.726971 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:15:01.726941    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod34ccf2cf_33e4_4856_a8c6_a2944f558cd8.slice/crio-89f10288b5e53d568884954a61bcdcf92f9b054bb4b8c2d631add8000db9b333 WatchSource:0}: Error finding container 89f10288b5e53d568884954a61bcdcf92f9b054bb4b8c2d631add8000db9b333: Status 404 returned error can't find the container with id 89f10288b5e53d568884954a61bcdcf92f9b054bb4b8c2d631add8000db9b333
May 06 17:15:02.003646 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:02.003609    2560 generic.go:358] "Generic (PLEG): container finished" podID="edccf5cc-c860-49e3-a67e-14cc757eb66e" containerID="a159497f942a11393533bf9d4f73cb3329abffa0da84c303fbaae5ec5dcba461" exitCode=0
May 06 17:15:02.003820 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:02.003674    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q" event={"ID":"edccf5cc-c860-49e3-a67e-14cc757eb66e","Type":"ContainerDied","Data":"a159497f942a11393533bf9d4f73cb3329abffa0da84c303fbaae5ec5dcba461"}
May 06 17:15:02.004793 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:02.004769    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-dvfp5" event={"ID":"34ccf2cf-33e4-4856-a8c6-a2944f558cd8","Type":"ContainerStarted","Data":"89f10288b5e53d568884954a61bcdcf92f9b054bb4b8c2d631add8000db9b333"}
May 06 17:15:03.009339 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:03.009305    2560 generic.go:358] "Generic (PLEG): container finished" podID="edccf5cc-c860-49e3-a67e-14cc757eb66e" containerID="84f06bf8396a4155700e11660fe89b88de50549a1e29ca37932b81bcb4258ae1" exitCode=0
May 06 17:15:03.009783 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:03.009347    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q" event={"ID":"edccf5cc-c860-49e3-a67e-14cc757eb66e","Type":"ContainerDied","Data":"84f06bf8396a4155700e11660fe89b88de50549a1e29ca37932b81bcb4258ae1"}
May 06 17:15:04.136718 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:04.136693    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:15:04.262881 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:04.262793    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gd64w\" (UniqueName: \"kubernetes.io/projected/edccf5cc-c860-49e3-a67e-14cc757eb66e-kube-api-access-gd64w\") pod \"edccf5cc-c860-49e3-a67e-14cc757eb66e\" (UID: \"edccf5cc-c860-49e3-a67e-14cc757eb66e\") "
May 06 17:15:04.262881 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:04.262872    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edccf5cc-c860-49e3-a67e-14cc757eb66e-bundle\") pod \"edccf5cc-c860-49e3-a67e-14cc757eb66e\" (UID: \"edccf5cc-c860-49e3-a67e-14cc757eb66e\") "
May 06 17:15:04.263103 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:04.262947    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edccf5cc-c860-49e3-a67e-14cc757eb66e-util\") pod \"edccf5cc-c860-49e3-a67e-14cc757eb66e\" (UID: \"edccf5cc-c860-49e3-a67e-14cc757eb66e\") "
May 06 17:15:04.267413 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:04.264107    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edccf5cc-c860-49e3-a67e-14cc757eb66e-bundle" (OuterVolumeSpecName: "bundle") pod "edccf5cc-c860-49e3-a67e-14cc757eb66e" (UID: "edccf5cc-c860-49e3-a67e-14cc757eb66e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:15:04.269514 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:04.269479    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/edccf5cc-c860-49e3-a67e-14cc757eb66e-kube-api-access-gd64w" (OuterVolumeSpecName: "kube-api-access-gd64w") pod "edccf5cc-c860-49e3-a67e-14cc757eb66e" (UID: "edccf5cc-c860-49e3-a67e-14cc757eb66e"). InnerVolumeSpecName "kube-api-access-gd64w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:15:04.270231 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:04.270165    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/edccf5cc-c860-49e3-a67e-14cc757eb66e-util" (OuterVolumeSpecName: "util") pod "edccf5cc-c860-49e3-a67e-14cc757eb66e" (UID: "edccf5cc-c860-49e3-a67e-14cc757eb66e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:15:04.364058 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:04.364023    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/edccf5cc-c860-49e3-a67e-14cc757eb66e-util\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:15:04.364058 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:04.364059    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gd64w\" (UniqueName: \"kubernetes.io/projected/edccf5cc-c860-49e3-a67e-14cc757eb66e-kube-api-access-gd64w\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:15:04.364292 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:04.364071    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/edccf5cc-c860-49e3-a67e-14cc757eb66e-bundle\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:15:05.018055 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.018018    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q" event={"ID":"edccf5cc-c860-49e3-a67e-14cc757eb66e","Type":"ContainerDied","Data":"da7e842a862b008934f8e893bd6e68e3b469b9ec0b4f777c7e5846ecf9643a04"}
May 06 17:15:05.018055 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.018058    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="da7e842a862b008934f8e893bd6e68e3b469b9ec0b4f777c7e5846ecf9643a04"
May 06 17:15:05.018314 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.018074    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/77defbb6647882b321b46d16bdefe62633c3425354d3d93571a1d6a87fqr97q"
May 06 17:15:05.696091 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.696055    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-9vhvz"]
May 06 17:15:05.696505 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.696381    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edccf5cc-c860-49e3-a67e-14cc757eb66e" containerName="extract"
May 06 17:15:05.696505 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.696394    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="edccf5cc-c860-49e3-a67e-14cc757eb66e" containerName="extract"
May 06 17:15:05.696505 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.696404    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edccf5cc-c860-49e3-a67e-14cc757eb66e" containerName="util"
May 06 17:15:05.696505 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.696409    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="edccf5cc-c860-49e3-a67e-14cc757eb66e" containerName="util"
May 06 17:15:05.696505 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.696415    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="edccf5cc-c860-49e3-a67e-14cc757eb66e" containerName="pull"
May 06 17:15:05.696505 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.696421    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="edccf5cc-c860-49e3-a67e-14cc757eb66e" containerName="pull"
May 06 17:15:05.696505 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.696493    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="edccf5cc-c860-49e3-a67e-14cc757eb66e" containerName="extract"
May 06 17:15:05.700469 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.700446    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz"
May 06 17:15:05.703165 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.703130    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"cert-manager\"/\"cert-manager-webhook-dockercfg-hnc9f\""
May 06 17:15:05.711057 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.711031    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-9vhvz"]
May 06 17:15:05.775765 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.775715    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c66843e-9ba1-4a52-856a-4e5f1428f4ec-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-9vhvz\" (UID: \"0c66843e-9ba1-4a52-856a-4e5f1428f4ec\") " pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz"
May 06 17:15:05.775967 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.775782    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jjg46\" (UniqueName: \"kubernetes.io/projected/0c66843e-9ba1-4a52-856a-4e5f1428f4ec-kube-api-access-jjg46\") pod \"cert-manager-webhook-587ccfb98-9vhvz\" (UID: \"0c66843e-9ba1-4a52-856a-4e5f1428f4ec\") " pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz"
May 06 17:15:05.877219 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.877176    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c66843e-9ba1-4a52-856a-4e5f1428f4ec-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-9vhvz\" (UID: \"0c66843e-9ba1-4a52-856a-4e5f1428f4ec\") " pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz"
May 06 17:15:05.877403 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.877225    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jjg46\" (UniqueName: \"kubernetes.io/projected/0c66843e-9ba1-4a52-856a-4e5f1428f4ec-kube-api-access-jjg46\") pod \"cert-manager-webhook-587ccfb98-9vhvz\" (UID: \"0c66843e-9ba1-4a52-856a-4e5f1428f4ec\") " pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz"
May 06 17:15:05.887264 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.887236    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/0c66843e-9ba1-4a52-856a-4e5f1428f4ec-bound-sa-token\") pod \"cert-manager-webhook-587ccfb98-9vhvz\" (UID: \"0c66843e-9ba1-4a52-856a-4e5f1428f4ec\") " pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz"
May 06 17:15:05.887420 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:05.887287    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jjg46\" (UniqueName: \"kubernetes.io/projected/0c66843e-9ba1-4a52-856a-4e5f1428f4ec-kube-api-access-jjg46\") pod \"cert-manager-webhook-587ccfb98-9vhvz\" (UID: \"0c66843e-9ba1-4a52-856a-4e5f1428f4ec\") " pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz"
May 06 17:15:06.011478 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:06.011390    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz"
May 06 17:15:06.258787 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:06.258755    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["cert-manager/cert-manager-webhook-587ccfb98-9vhvz"]
May 06 17:15:06.262529 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:15:06.262501    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0c66843e_9ba1_4a52_856a_4e5f1428f4ec.slice/crio-09dc8c485621b3ac652294ec6cb32325d701475300c05e3b96722d30790db4f5 WatchSource:0}: Error finding container 09dc8c485621b3ac652294ec6cb32325d701475300c05e3b96722d30790db4f5: Status 404 returned error can't find the container with id 09dc8c485621b3ac652294ec6cb32325d701475300c05e3b96722d30790db4f5
May 06 17:15:07.025137 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:07.025096    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz" event={"ID":"0c66843e-9ba1-4a52-856a-4e5f1428f4ec","Type":"ContainerStarted","Data":"427bc3b815f80cfe61c7f1eff11e452816d861cb5d1f42e602de62fcde7ecab3"}
May 06 17:15:07.025137 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:07.025134    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz" event={"ID":"0c66843e-9ba1-4a52-856a-4e5f1428f4ec","Type":"ContainerStarted","Data":"09dc8c485621b3ac652294ec6cb32325d701475300c05e3b96722d30790db4f5"}
May 06 17:15:07.025623 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:07.025181    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz"
May 06 17:15:07.026387 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:07.026364    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="cert-manager/cert-manager-cainjector-68b757865b-dvfp5" event={"ID":"34ccf2cf-33e4-4856-a8c6-a2944f558cd8","Type":"ContainerStarted","Data":"439206e3da68c80b24ccf1da0db429e5b996981bc8bd5ed89733e34b1d0c6e28"}
May 06 17:15:07.043166 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:07.043108    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz" podStartSLOduration=2.043095072 podStartE2EDuration="2.043095072s" podCreationTimestamp="2026-05-06 17:15:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-06 17:15:07.041020965 +0000 UTC m=+278.484843412" watchObservedRunningTime="2026-05-06 17:15:07.043095072 +0000 UTC m=+278.486917517"
May 06 17:15:07.057685 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:07.057632    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="cert-manager/cert-manager-cainjector-68b757865b-dvfp5" podStartSLOduration=1.588684244 podStartE2EDuration="6.057614328s" podCreationTimestamp="2026-05-06 17:15:01 +0000 UTC" firstStartedPulling="2026-05-06 17:15:01.728772278 +0000 UTC m=+273.172594703" lastFinishedPulling="2026-05-06 17:15:06.19770235 +0000 UTC m=+277.641524787" observedRunningTime="2026-05-06 17:15:07.056727019 +0000 UTC m=+278.500549465" watchObservedRunningTime="2026-05-06 17:15:07.057614328 +0000 UTC m=+278.501436777"
May 06 17:15:13.032040 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:13.032006    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="cert-manager/cert-manager-webhook-587ccfb98-9vhvz"
May 06 17:15:18.022760 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.022720    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"]
May 06 17:15:18.026113 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.026092    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:18.028427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.028407    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tpsdd\""
May 06 17:15:18.028427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.028422    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 06 17:15:18.028535 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.028428    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 06 17:15:18.033057 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.033034    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"]
May 06 17:15:18.172603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.172566    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee87b574-adde-47e6-ba73-891c2f189b53-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws\" (UID: \"ee87b574-adde-47e6-ba73-891c2f189b53\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:18.172603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.172607    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee87b574-adde-47e6-ba73-891c2f189b53-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws\" (UID: \"ee87b574-adde-47e6-ba73-891c2f189b53\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:18.172800 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.172634    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hc4d\" (UniqueName: \"kubernetes.io/projected/ee87b574-adde-47e6-ba73-891c2f189b53-kube-api-access-9hc4d\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws\" (UID: \"ee87b574-adde-47e6-ba73-891c2f189b53\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:18.273634 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.273546    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee87b574-adde-47e6-ba73-891c2f189b53-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws\" (UID: \"ee87b574-adde-47e6-ba73-891c2f189b53\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:18.273634 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.273583    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee87b574-adde-47e6-ba73-891c2f189b53-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws\" (UID: \"ee87b574-adde-47e6-ba73-891c2f189b53\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:18.273844 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.273640    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9hc4d\" (UniqueName: \"kubernetes.io/projected/ee87b574-adde-47e6-ba73-891c2f189b53-kube-api-access-9hc4d\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws\" (UID: \"ee87b574-adde-47e6-ba73-891c2f189b53\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:18.273951 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.273931    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee87b574-adde-47e6-ba73-891c2f189b53-bundle\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws\" (UID: \"ee87b574-adde-47e6-ba73-891c2f189b53\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:18.274020 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.273983    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee87b574-adde-47e6-ba73-891c2f189b53-util\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws\" (UID: \"ee87b574-adde-47e6-ba73-891c2f189b53\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:18.299844 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.299808    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hc4d\" (UniqueName: \"kubernetes.io/projected/ee87b574-adde-47e6-ba73-891c2f189b53-kube-api-access-9hc4d\") pod \"3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws\" (UID: \"ee87b574-adde-47e6-ba73-891c2f189b53\") " pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:18.336511 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.336474    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:18.462251 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:18.462214    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"]
May 06 17:15:18.464866 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:15:18.464837    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podee87b574_adde_47e6_ba73_891c2f189b53.slice/crio-f95e2c1a4983a63c9109a3545e4eb3aa23207966e67837e94cccca0011fc936d WatchSource:0}: Error finding container f95e2c1a4983a63c9109a3545e4eb3aa23207966e67837e94cccca0011fc936d: Status 404 returned error can't find the container with id f95e2c1a4983a63c9109a3545e4eb3aa23207966e67837e94cccca0011fc936d
May 06 17:15:19.067278 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:19.067239    2560 generic.go:358] "Generic (PLEG): container finished" podID="ee87b574-adde-47e6-ba73-891c2f189b53" containerID="260320e4c018bf6fdf5eb7bb013efa76cb6c044f2deb7c2265eb3ecb874776da" exitCode=0
May 06 17:15:19.067668 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:19.067322    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws" event={"ID":"ee87b574-adde-47e6-ba73-891c2f189b53","Type":"ContainerDied","Data":"260320e4c018bf6fdf5eb7bb013efa76cb6c044f2deb7c2265eb3ecb874776da"}
May 06 17:15:19.067668 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:19.067361    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws" event={"ID":"ee87b574-adde-47e6-ba73-891c2f189b53","Type":"ContainerStarted","Data":"f95e2c1a4983a63c9109a3545e4eb3aa23207966e67837e94cccca0011fc936d"}
May 06 17:15:20.071565 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:20.071530    2560 generic.go:358] "Generic (PLEG): container finished" podID="ee87b574-adde-47e6-ba73-891c2f189b53" containerID="c8f44d0154b5e947d491af243ed032d579c971c5395cd1aee14e6f6fb77ad85d" exitCode=0
May 06 17:15:20.072033 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:20.071619    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws" event={"ID":"ee87b574-adde-47e6-ba73-891c2f189b53","Type":"ContainerDied","Data":"c8f44d0154b5e947d491af243ed032d579c971c5395cd1aee14e6f6fb77ad85d"}
May 06 17:15:21.077063 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:21.077022    2560 generic.go:358] "Generic (PLEG): container finished" podID="ee87b574-adde-47e6-ba73-891c2f189b53" containerID="eaf0a345cd0d92a3120f3e2a1ffa5d13bba5f4c8a0cd841313f41060efde0236" exitCode=0
May 06 17:15:21.077507 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:21.077071    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws" event={"ID":"ee87b574-adde-47e6-ba73-891c2f189b53","Type":"ContainerDied","Data":"eaf0a345cd0d92a3120f3e2a1ffa5d13bba5f4c8a0cd841313f41060efde0236"}
May 06 17:15:22.202329 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:22.202300    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:22.309218 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:22.309133    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee87b574-adde-47e6-ba73-891c2f189b53-util\") pod \"ee87b574-adde-47e6-ba73-891c2f189b53\" (UID: \"ee87b574-adde-47e6-ba73-891c2f189b53\") "
May 06 17:15:22.309398 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:22.309277    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee87b574-adde-47e6-ba73-891c2f189b53-bundle\") pod \"ee87b574-adde-47e6-ba73-891c2f189b53\" (UID: \"ee87b574-adde-47e6-ba73-891c2f189b53\") "
May 06 17:15:22.309398 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:22.309309    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9hc4d\" (UniqueName: \"kubernetes.io/projected/ee87b574-adde-47e6-ba73-891c2f189b53-kube-api-access-9hc4d\") pod \"ee87b574-adde-47e6-ba73-891c2f189b53\" (UID: \"ee87b574-adde-47e6-ba73-891c2f189b53\") "
May 06 17:15:22.309969 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:22.309943    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee87b574-adde-47e6-ba73-891c2f189b53-bundle" (OuterVolumeSpecName: "bundle") pod "ee87b574-adde-47e6-ba73-891c2f189b53" (UID: "ee87b574-adde-47e6-ba73-891c2f189b53"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:15:22.311400 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:22.311373    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ee87b574-adde-47e6-ba73-891c2f189b53-kube-api-access-9hc4d" (OuterVolumeSpecName: "kube-api-access-9hc4d") pod "ee87b574-adde-47e6-ba73-891c2f189b53" (UID: "ee87b574-adde-47e6-ba73-891c2f189b53"). InnerVolumeSpecName "kube-api-access-9hc4d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:15:22.314883 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:22.314840    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ee87b574-adde-47e6-ba73-891c2f189b53-util" (OuterVolumeSpecName: "util") pod "ee87b574-adde-47e6-ba73-891c2f189b53" (UID: "ee87b574-adde-47e6-ba73-891c2f189b53"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:15:22.410737 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:22.410650    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/ee87b574-adde-47e6-ba73-891c2f189b53-util\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:15:22.410737 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:22.410683    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/ee87b574-adde-47e6-ba73-891c2f189b53-bundle\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:15:22.410737 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:22.410695    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9hc4d\" (UniqueName: \"kubernetes.io/projected/ee87b574-adde-47e6-ba73-891c2f189b53-kube-api-access-9hc4d\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:15:23.087885 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:23.087847    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws" event={"ID":"ee87b574-adde-47e6-ba73-891c2f189b53","Type":"ContainerDied","Data":"f95e2c1a4983a63c9109a3545e4eb3aa23207966e67837e94cccca0011fc936d"}
May 06 17:15:23.087885 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:23.087880    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="f95e2c1a4983a63c9109a3545e4eb3aa23207966e67837e94cccca0011fc936d"
May 06 17:15:23.088089 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:23.087904    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/3081035efbc3f76b8ae1b663abbd2a6bc5b4896fe94818011b247755c5hccws"
May 06 17:15:24.561194 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.561142    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"]
May 06 17:15:24.561691 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.561556    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee87b574-adde-47e6-ba73-891c2f189b53" containerName="extract"
May 06 17:15:24.561691 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.561574    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee87b574-adde-47e6-ba73-891c2f189b53" containerName="extract"
May 06 17:15:24.561691 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.561601    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee87b574-adde-47e6-ba73-891c2f189b53" containerName="util"
May 06 17:15:24.561691 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.561610    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee87b574-adde-47e6-ba73-891c2f189b53" containerName="util"
May 06 17:15:24.561691 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.561620    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ee87b574-adde-47e6-ba73-891c2f189b53" containerName="pull"
May 06 17:15:24.561691 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.561629    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="ee87b574-adde-47e6-ba73-891c2f189b53" containerName="pull"
May 06 17:15:24.561968 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.561712    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="ee87b574-adde-47e6-ba73-891c2f189b53" containerName="extract"
May 06 17:15:24.565801 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.565780    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.569096 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.569074    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"lws-controller-manager-dockercfg-b99g6\""
May 06 17:15:24.569238 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.569096    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"lws-manager-config\""
May 06 17:15:24.569238 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.569114    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"webhook-server-cert\""
May 06 17:15:24.569238 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.569083    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"kube-root-ca.crt\""
May 06 17:15:24.569238 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.569081    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-lws-operator\"/\"metrics-server-cert\""
May 06 17:15:24.569238 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.569105    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-lws-operator\"/\"openshift-service-ca.crt\""
May 06 17:15:24.573543 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.573523    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"]
May 06 17:15:24.730872 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.730811    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a78bdda-da4e-4f60-8738-055fef49eecb-cert\") pod \"lws-controller-manager-7fd7474bdd-dsz2k\" (UID: \"3a78bdda-da4e-4f60-8738-055fef49eecb\") " pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.730872 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.730867    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3a78bdda-da4e-4f60-8738-055fef49eecb-manager-config\") pod \"lws-controller-manager-7fd7474bdd-dsz2k\" (UID: \"3a78bdda-da4e-4f60-8738-055fef49eecb\") " pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.730872 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.730890    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a78bdda-da4e-4f60-8738-055fef49eecb-metrics-cert\") pod \"lws-controller-manager-7fd7474bdd-dsz2k\" (UID: \"3a78bdda-da4e-4f60-8738-055fef49eecb\") " pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.731132 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.730968    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7kpfk\" (UniqueName: \"kubernetes.io/projected/3a78bdda-da4e-4f60-8738-055fef49eecb-kube-api-access-7kpfk\") pod \"lws-controller-manager-7fd7474bdd-dsz2k\" (UID: \"3a78bdda-da4e-4f60-8738-055fef49eecb\") " pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.831978 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.831902    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a78bdda-da4e-4f60-8738-055fef49eecb-metrics-cert\") pod \"lws-controller-manager-7fd7474bdd-dsz2k\" (UID: \"3a78bdda-da4e-4f60-8738-055fef49eecb\") " pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.831978 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.831941    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7kpfk\" (UniqueName: \"kubernetes.io/projected/3a78bdda-da4e-4f60-8738-055fef49eecb-kube-api-access-7kpfk\") pod \"lws-controller-manager-7fd7474bdd-dsz2k\" (UID: \"3a78bdda-da4e-4f60-8738-055fef49eecb\") " pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.832236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.831993    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a78bdda-da4e-4f60-8738-055fef49eecb-cert\") pod \"lws-controller-manager-7fd7474bdd-dsz2k\" (UID: \"3a78bdda-da4e-4f60-8738-055fef49eecb\") " pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.832236 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.832013    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3a78bdda-da4e-4f60-8738-055fef49eecb-manager-config\") pod \"lws-controller-manager-7fd7474bdd-dsz2k\" (UID: \"3a78bdda-da4e-4f60-8738-055fef49eecb\") " pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.832795 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.832765    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"manager-config\" (UniqueName: \"kubernetes.io/configmap/3a78bdda-da4e-4f60-8738-055fef49eecb-manager-config\") pod \"lws-controller-manager-7fd7474bdd-dsz2k\" (UID: \"3a78bdda-da4e-4f60-8738-055fef49eecb\") " pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.834502 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.834475    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/3a78bdda-da4e-4f60-8738-055fef49eecb-cert\") pod \"lws-controller-manager-7fd7474bdd-dsz2k\" (UID: \"3a78bdda-da4e-4f60-8738-055fef49eecb\") " pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.834589 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.834522    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-cert\" (UniqueName: \"kubernetes.io/secret/3a78bdda-da4e-4f60-8738-055fef49eecb-metrics-cert\") pod \"lws-controller-manager-7fd7474bdd-dsz2k\" (UID: \"3a78bdda-da4e-4f60-8738-055fef49eecb\") " pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.849460 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.849440    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7kpfk\" (UniqueName: \"kubernetes.io/projected/3a78bdda-da4e-4f60-8738-055fef49eecb-kube-api-access-7kpfk\") pod \"lws-controller-manager-7fd7474bdd-dsz2k\" (UID: \"3a78bdda-da4e-4f60-8738-055fef49eecb\") " pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:24.875249 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:24.875221    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:25.007304 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:25.007162    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"]
May 06 17:15:25.009859 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:15:25.009828    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3a78bdda_da4e_4f60_8738_055fef49eecb.slice/crio-9bd0f2d77bbcf9c2834f7d812d4b2c54b44a7230db89ff2a6845b8782e5d62b2 WatchSource:0}: Error finding container 9bd0f2d77bbcf9c2834f7d812d4b2c54b44a7230db89ff2a6845b8782e5d62b2: Status 404 returned error can't find the container with id 9bd0f2d77bbcf9c2834f7d812d4b2c54b44a7230db89ff2a6845b8782e5d62b2
May 06 17:15:25.095239 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:25.095138    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k" event={"ID":"3a78bdda-da4e-4f60-8738-055fef49eecb","Type":"ContainerStarted","Data":"9bd0f2d77bbcf9c2834f7d812d4b2c54b44a7230db89ff2a6845b8782e5d62b2"}
May 06 17:15:27.103072 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:27.102966    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k" event={"ID":"3a78bdda-da4e-4f60-8738-055fef49eecb","Type":"ContainerStarted","Data":"16edacbecaf90490b292956b288de79cb09f5a4ffd09fc7d4ea06443d77a9a4d"}
May 06 17:15:27.103072 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:27.103055    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:27.121289 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:27.121246    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k" podStartSLOduration=1.321390879 podStartE2EDuration="3.121232484s" podCreationTimestamp="2026-05-06 17:15:24 +0000 UTC" firstStartedPulling="2026-05-06 17:15:25.011733269 +0000 UTC m=+296.455555694" lastFinishedPulling="2026-05-06 17:15:26.811574871 +0000 UTC m=+298.255397299" observedRunningTime="2026-05-06 17:15:27.119594882 +0000 UTC m=+298.563417330" watchObservedRunningTime="2026-05-06 17:15:27.121232484 +0000 UTC m=+298.565054930"
May 06 17:15:29.014346 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:29.014320    2560 kubelet.go:1628] "Image garbage collection succeeded"
May 06 17:15:35.240422 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.240387    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"]
May 06 17:15:35.243547 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.243531    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:35.245861 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.245837    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 06 17:15:35.245993 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.245837    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tpsdd\""
May 06 17:15:35.246590 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.246576    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 06 17:15:35.253017 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.252997    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"]
May 06 17:15:35.316416 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.316387    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78813d96-5bef-4cde-9d75-6158bf1ad4b6-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9\" (UID: \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:35.316562 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.316425    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78813d96-5bef-4cde-9d75-6158bf1ad4b6-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9\" (UID: \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:35.316562 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.316497    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nh8ps\" (UniqueName: \"kubernetes.io/projected/78813d96-5bef-4cde-9d75-6158bf1ad4b6-kube-api-access-nh8ps\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9\" (UID: \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:35.417008 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.416971    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nh8ps\" (UniqueName: \"kubernetes.io/projected/78813d96-5bef-4cde-9d75-6158bf1ad4b6-kube-api-access-nh8ps\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9\" (UID: \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:35.417185 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.417042    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78813d96-5bef-4cde-9d75-6158bf1ad4b6-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9\" (UID: \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:35.417185 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.417063    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78813d96-5bef-4cde-9d75-6158bf1ad4b6-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9\" (UID: \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:35.417449 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.417434    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78813d96-5bef-4cde-9d75-6158bf1ad4b6-bundle\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9\" (UID: \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:35.417486 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.417455    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78813d96-5bef-4cde-9d75-6158bf1ad4b6-util\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9\" (UID: \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:35.424730 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.424706    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nh8ps\" (UniqueName: \"kubernetes.io/projected/78813d96-5bef-4cde-9d75-6158bf1ad4b6-kube-api-access-nh8ps\") pod \"f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9\" (UID: \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\") " pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:35.447681 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.447649    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"]
May 06 17:15:35.449872 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.449845    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:35.452437 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.452416    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-webhook-cert\""
May 06 17:15:35.452889 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.452873    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-service-cert\""
May 06 17:15:35.453052 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.453036    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"opendatahub-operator-controller-manager-dockercfg-spdkz\""
May 06 17:15:35.453128 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.453115    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"openshift-service-ca.crt\""
May 06 17:15:35.453269 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.453257    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"kube-root-ca.crt\""
May 06 17:15:35.467161 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.467111    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"]
May 06 17:15:35.518238 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.518130    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cfe6f3cf-2088-4a20-a377-a3eb27895bee-apiservice-cert\") pod \"opendatahub-operator-controller-manager-698574c4f-tnxwn\" (UID: \"cfe6f3cf-2088-4a20-a377-a3eb27895bee\") " pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:35.518238 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.518189    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfe6f3cf-2088-4a20-a377-a3eb27895bee-webhook-cert\") pod \"opendatahub-operator-controller-manager-698574c4f-tnxwn\" (UID: \"cfe6f3cf-2088-4a20-a377-a3eb27895bee\") " pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:35.518419 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.518239    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mvnrr\" (UniqueName: \"kubernetes.io/projected/cfe6f3cf-2088-4a20-a377-a3eb27895bee-kube-api-access-mvnrr\") pod \"opendatahub-operator-controller-manager-698574c4f-tnxwn\" (UID: \"cfe6f3cf-2088-4a20-a377-a3eb27895bee\") " pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:35.553053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.553022    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:35.618903 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.618866    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mvnrr\" (UniqueName: \"kubernetes.io/projected/cfe6f3cf-2088-4a20-a377-a3eb27895bee-kube-api-access-mvnrr\") pod \"opendatahub-operator-controller-manager-698574c4f-tnxwn\" (UID: \"cfe6f3cf-2088-4a20-a377-a3eb27895bee\") " pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:35.619088 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.618989    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cfe6f3cf-2088-4a20-a377-a3eb27895bee-apiservice-cert\") pod \"opendatahub-operator-controller-manager-698574c4f-tnxwn\" (UID: \"cfe6f3cf-2088-4a20-a377-a3eb27895bee\") " pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:35.619088 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.619015    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfe6f3cf-2088-4a20-a377-a3eb27895bee-webhook-cert\") pod \"opendatahub-operator-controller-manager-698574c4f-tnxwn\" (UID: \"cfe6f3cf-2088-4a20-a377-a3eb27895bee\") " pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:35.621980 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.621941    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"apiservice-cert\" (UniqueName: \"kubernetes.io/secret/cfe6f3cf-2088-4a20-a377-a3eb27895bee-apiservice-cert\") pod \"opendatahub-operator-controller-manager-698574c4f-tnxwn\" (UID: \"cfe6f3cf-2088-4a20-a377-a3eb27895bee\") " pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:35.622590 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.622564    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"webhook-cert\" (UniqueName: \"kubernetes.io/secret/cfe6f3cf-2088-4a20-a377-a3eb27895bee-webhook-cert\") pod \"opendatahub-operator-controller-manager-698574c4f-tnxwn\" (UID: \"cfe6f3cf-2088-4a20-a377-a3eb27895bee\") " pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:35.638767 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.638743    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mvnrr\" (UniqueName: \"kubernetes.io/projected/cfe6f3cf-2088-4a20-a377-a3eb27895bee-kube-api-access-mvnrr\") pod \"opendatahub-operator-controller-manager-698574c4f-tnxwn\" (UID: \"cfe6f3cf-2088-4a20-a377-a3eb27895bee\") " pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:35.682758 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.682643    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"]
May 06 17:15:35.684867 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:15:35.684828    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod78813d96_5bef_4cde_9d75_6158bf1ad4b6.slice/crio-5fb3a79e5fbdcfb14115baebc49e1538071d99407b699707d54ab0e69f554319 WatchSource:0}: Error finding container 5fb3a79e5fbdcfb14115baebc49e1538071d99407b699707d54ab0e69f554319: Status 404 returned error can't find the container with id 5fb3a79e5fbdcfb14115baebc49e1538071d99407b699707d54ab0e69f554319
May 06 17:15:35.686675 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.686660    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 06 17:15:35.761199 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.761143    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:35.921879 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:35.921812    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"]
May 06 17:15:35.924648 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:15:35.924618    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcfe6f3cf_2088_4a20_a377_a3eb27895bee.slice/crio-6afd9f8278594a1d47521c55c76eb923cb3cff50e604eb8ad84e23052da2c380 WatchSource:0}: Error finding container 6afd9f8278594a1d47521c55c76eb923cb3cff50e604eb8ad84e23052da2c380: Status 404 returned error can't find the container with id 6afd9f8278594a1d47521c55c76eb923cb3cff50e604eb8ad84e23052da2c380
May 06 17:15:36.130925 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:36.130839    2560 generic.go:358] "Generic (PLEG): container finished" podID="78813d96-5bef-4cde-9d75-6158bf1ad4b6" containerID="f76e6c331040c33c0b13eba46561316468dab43511cf38acad5b7878ea8e063e" exitCode=0
May 06 17:15:36.131095 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:36.130918    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9" event={"ID":"78813d96-5bef-4cde-9d75-6158bf1ad4b6","Type":"ContainerDied","Data":"f76e6c331040c33c0b13eba46561316468dab43511cf38acad5b7878ea8e063e"}
May 06 17:15:36.131095 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:36.130952    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9" event={"ID":"78813d96-5bef-4cde-9d75-6158bf1ad4b6","Type":"ContainerStarted","Data":"5fb3a79e5fbdcfb14115baebc49e1538071d99407b699707d54ab0e69f554319"}
May 06 17:15:36.132235 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:36.132209    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn" event={"ID":"cfe6f3cf-2088-4a20-a377-a3eb27895bee","Type":"ContainerStarted","Data":"6afd9f8278594a1d47521c55c76eb923cb3cff50e604eb8ad84e23052da2c380"}
May 06 17:15:37.137838 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:37.137725    2560 generic.go:358] "Generic (PLEG): container finished" podID="78813d96-5bef-4cde-9d75-6158bf1ad4b6" containerID="f8158bb71f29cae7a8901ab954a0cac1d8ea497056ae003e27ada10d26c5d014" exitCode=0
May 06 17:15:37.138287 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:37.137831    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9" event={"ID":"78813d96-5bef-4cde-9d75-6158bf1ad4b6","Type":"ContainerDied","Data":"f8158bb71f29cae7a8901ab954a0cac1d8ea497056ae003e27ada10d26c5d014"}
May 06 17:15:38.108783 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:38.108750    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-lws-operator/lws-controller-manager-7fd7474bdd-dsz2k"
May 06 17:15:38.143303 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:38.143269    2560 generic.go:358] "Generic (PLEG): container finished" podID="78813d96-5bef-4cde-9d75-6158bf1ad4b6" containerID="23a3194b83a40df4491d1a225b11589a13d469c385a116c957f8f2e54bc78bc9" exitCode=0
May 06 17:15:38.143748 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:38.143352    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9" event={"ID":"78813d96-5bef-4cde-9d75-6158bf1ad4b6","Type":"ContainerDied","Data":"23a3194b83a40df4491d1a225b11589a13d469c385a116c957f8f2e54bc78bc9"}
May 06 17:15:39.147567 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.147473    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn" event={"ID":"cfe6f3cf-2088-4a20-a377-a3eb27895bee","Type":"ContainerStarted","Data":"ab7451e9902bbea1470c7ca5c751c9ccd27f358b7a3f78ea4babbba712feb89f"}
May 06 17:15:39.147968 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.147680    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:39.185652 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.185604    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn" podStartSLOduration=1.334351278 podStartE2EDuration="4.185587729s" podCreationTimestamp="2026-05-06 17:15:35 +0000 UTC" firstStartedPulling="2026-05-06 17:15:35.926440963 +0000 UTC m=+307.370263387" lastFinishedPulling="2026-05-06 17:15:38.777677409 +0000 UTC m=+310.221499838" observedRunningTime="2026-05-06 17:15:39.184902745 +0000 UTC m=+310.628725193" watchObservedRunningTime="2026-05-06 17:15:39.185587729 +0000 UTC m=+310.629410176"
May 06 17:15:39.267562 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.267534    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:39.346980 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.346940    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78813d96-5bef-4cde-9d75-6158bf1ad4b6-util\") pod \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\" (UID: \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\") "
May 06 17:15:39.347179 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.347008    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78813d96-5bef-4cde-9d75-6158bf1ad4b6-bundle\") pod \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\" (UID: \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\") "
May 06 17:15:39.347179 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.347037    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nh8ps\" (UniqueName: \"kubernetes.io/projected/78813d96-5bef-4cde-9d75-6158bf1ad4b6-kube-api-access-nh8ps\") pod \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\" (UID: \"78813d96-5bef-4cde-9d75-6158bf1ad4b6\") "
May 06 17:15:39.347734 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.347708    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78813d96-5bef-4cde-9d75-6158bf1ad4b6-bundle" (OuterVolumeSpecName: "bundle") pod "78813d96-5bef-4cde-9d75-6158bf1ad4b6" (UID: "78813d96-5bef-4cde-9d75-6158bf1ad4b6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:15:39.349131 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.349102    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/78813d96-5bef-4cde-9d75-6158bf1ad4b6-kube-api-access-nh8ps" (OuterVolumeSpecName: "kube-api-access-nh8ps") pod "78813d96-5bef-4cde-9d75-6158bf1ad4b6" (UID: "78813d96-5bef-4cde-9d75-6158bf1ad4b6"). InnerVolumeSpecName "kube-api-access-nh8ps". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:15:39.352191 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.352168    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/78813d96-5bef-4cde-9d75-6158bf1ad4b6-util" (OuterVolumeSpecName: "util") pod "78813d96-5bef-4cde-9d75-6158bf1ad4b6" (UID: "78813d96-5bef-4cde-9d75-6158bf1ad4b6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:15:39.447589 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.447550    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/78813d96-5bef-4cde-9d75-6158bf1ad4b6-util\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:15:39.447589 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.447581    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/78813d96-5bef-4cde-9d75-6158bf1ad4b6-bundle\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:15:39.447589 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:39.447591    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nh8ps\" (UniqueName: \"kubernetes.io/projected/78813d96-5bef-4cde-9d75-6158bf1ad4b6-kube-api-access-nh8ps\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:15:40.152258 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:40.152219    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9" event={"ID":"78813d96-5bef-4cde-9d75-6158bf1ad4b6","Type":"ContainerDied","Data":"5fb3a79e5fbdcfb14115baebc49e1538071d99407b699707d54ab0e69f554319"}
May 06 17:15:40.152258 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:40.152261    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="5fb3a79e5fbdcfb14115baebc49e1538071d99407b699707d54ab0e69f554319"
May 06 17:15:40.152664 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:40.152232    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/f77c1a82ade775f7608969957d57ee0b8db93eeada9825bd6f7f7156c98n9p9"
May 06 17:15:50.155026 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:50.154993    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/opendatahub-operator-controller-manager-698574c4f-tnxwn"
May 06 17:15:55.570747 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.570706    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"]
May 06 17:15:55.571120 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.571043    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78813d96-5bef-4cde-9d75-6158bf1ad4b6" containerName="pull"
May 06 17:15:55.571120 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.571055    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="78813d96-5bef-4cde-9d75-6158bf1ad4b6" containerName="pull"
May 06 17:15:55.571120 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.571067    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78813d96-5bef-4cde-9d75-6158bf1ad4b6" containerName="util"
May 06 17:15:55.571120 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.571072    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="78813d96-5bef-4cde-9d75-6158bf1ad4b6" containerName="util"
May 06 17:15:55.571120 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.571082    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="78813d96-5bef-4cde-9d75-6158bf1ad4b6" containerName="extract"
May 06 17:15:55.571120 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.571087    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="78813d96-5bef-4cde-9d75-6158bf1ad4b6" containerName="extract"
May 06 17:15:55.571318 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.571138    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="78813d96-5bef-4cde-9d75-6158bf1ad4b6" containerName="extract"
May 06 17:15:55.573273 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.573256    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:15:55.575607 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.575579    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tpsdd\""
May 06 17:15:55.575742 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.575637    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 06 17:15:55.575742 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.575669    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 06 17:15:55.581734 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.581712    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"]
May 06 17:15:55.675053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.675007    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35686144-565a-4d97-b0c8-1fdd9dfa71e8-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn\" (UID: \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:15:55.675053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.675060    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4cjzq\" (UniqueName: \"kubernetes.io/projected/35686144-565a-4d97-b0c8-1fdd9dfa71e8-kube-api-access-4cjzq\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn\" (UID: \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:15:55.675357 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.675124    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35686144-565a-4d97-b0c8-1fdd9dfa71e8-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn\" (UID: \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:15:55.776197 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.776137    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4cjzq\" (UniqueName: \"kubernetes.io/projected/35686144-565a-4d97-b0c8-1fdd9dfa71e8-kube-api-access-4cjzq\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn\" (UID: \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:15:55.776360 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.776227    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35686144-565a-4d97-b0c8-1fdd9dfa71e8-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn\" (UID: \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:15:55.776360 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.776257    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35686144-565a-4d97-b0c8-1fdd9dfa71e8-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn\" (UID: \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:15:55.776602 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.776581    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35686144-565a-4d97-b0c8-1fdd9dfa71e8-bundle\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn\" (UID: \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:15:55.776638 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.776605    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35686144-565a-4d97-b0c8-1fdd9dfa71e8-util\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn\" (UID: \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:15:55.784024 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.783989    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4cjzq\" (UniqueName: \"kubernetes.io/projected/35686144-565a-4d97-b0c8-1fdd9dfa71e8-kube-api-access-4cjzq\") pod \"4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn\" (UID: \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\") " pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:15:55.883017 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:55.882935    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:15:56.012769 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:56.012743    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"]
May 06 17:15:56.014735 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:15:56.014703    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod35686144_565a_4d97_b0c8_1fdd9dfa71e8.slice/crio-70f87c18da221d5344028efa8e3098e0e8692d8be526e85a2faaed6b10dcf277 WatchSource:0}: Error finding container 70f87c18da221d5344028efa8e3098e0e8692d8be526e85a2faaed6b10dcf277: Status 404 returned error can't find the container with id 70f87c18da221d5344028efa8e3098e0e8692d8be526e85a2faaed6b10dcf277
May 06 17:15:56.205207 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:56.205172    2560 generic.go:358] "Generic (PLEG): container finished" podID="35686144-565a-4d97-b0c8-1fdd9dfa71e8" containerID="258f4b4ef6f5323fca88d1f44fe7448b2b758ffbc76f4196f07713b77b7dfd34" exitCode=0
May 06 17:15:56.205372 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:56.205221    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn" event={"ID":"35686144-565a-4d97-b0c8-1fdd9dfa71e8","Type":"ContainerDied","Data":"258f4b4ef6f5323fca88d1f44fe7448b2b758ffbc76f4196f07713b77b7dfd34"}
May 06 17:15:56.205372 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:56.205243    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn" event={"ID":"35686144-565a-4d97-b0c8-1fdd9dfa71e8","Type":"ContainerStarted","Data":"70f87c18da221d5344028efa8e3098e0e8692d8be526e85a2faaed6b10dcf277"}
May 06 17:15:57.210283 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:57.210249    2560 generic.go:358] "Generic (PLEG): container finished" podID="35686144-565a-4d97-b0c8-1fdd9dfa71e8" containerID="28405c50a977b47c702386aa627351c59903d9f18fe9e590bb117517694a1451" exitCode=0
May 06 17:15:57.210711 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:57.210331    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn" event={"ID":"35686144-565a-4d97-b0c8-1fdd9dfa71e8","Type":"ContainerDied","Data":"28405c50a977b47c702386aa627351c59903d9f18fe9e590bb117517694a1451"}
May 06 17:15:58.215389 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:58.215351    2560 generic.go:358] "Generic (PLEG): container finished" podID="35686144-565a-4d97-b0c8-1fdd9dfa71e8" containerID="76adc5901bb6c3250220e9feda8619f9dc7896b5b9f4887b410bbf43c176690b" exitCode=0
May 06 17:15:58.215389 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:58.215389    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn" event={"ID":"35686144-565a-4d97-b0c8-1fdd9dfa71e8","Type":"ContainerDied","Data":"76adc5901bb6c3250220e9feda8619f9dc7896b5b9f4887b410bbf43c176690b"}
May 06 17:15:59.343675 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:59.343649    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:15:59.405397 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:59.405361    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4cjzq\" (UniqueName: \"kubernetes.io/projected/35686144-565a-4d97-b0c8-1fdd9dfa71e8-kube-api-access-4cjzq\") pod \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\" (UID: \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\") "
May 06 17:15:59.405588 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:59.405405    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35686144-565a-4d97-b0c8-1fdd9dfa71e8-util\") pod \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\" (UID: \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\") "
May 06 17:15:59.405588 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:59.405454    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35686144-565a-4d97-b0c8-1fdd9dfa71e8-bundle\") pod \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\" (UID: \"35686144-565a-4d97-b0c8-1fdd9dfa71e8\") "
May 06 17:15:59.406319 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:59.406294    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35686144-565a-4d97-b0c8-1fdd9dfa71e8-bundle" (OuterVolumeSpecName: "bundle") pod "35686144-565a-4d97-b0c8-1fdd9dfa71e8" (UID: "35686144-565a-4d97-b0c8-1fdd9dfa71e8"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:15:59.407709 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:59.407684    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/35686144-565a-4d97-b0c8-1fdd9dfa71e8-kube-api-access-4cjzq" (OuterVolumeSpecName: "kube-api-access-4cjzq") pod "35686144-565a-4d97-b0c8-1fdd9dfa71e8" (UID: "35686144-565a-4d97-b0c8-1fdd9dfa71e8"). InnerVolumeSpecName "kube-api-access-4cjzq". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:15:59.411114 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:59.411092    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/35686144-565a-4d97-b0c8-1fdd9dfa71e8-util" (OuterVolumeSpecName: "util") pod "35686144-565a-4d97-b0c8-1fdd9dfa71e8" (UID: "35686144-565a-4d97-b0c8-1fdd9dfa71e8"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:15:59.506612 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:59.506527    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/35686144-565a-4d97-b0c8-1fdd9dfa71e8-bundle\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:15:59.506612 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:59.506559    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4cjzq\" (UniqueName: \"kubernetes.io/projected/35686144-565a-4d97-b0c8-1fdd9dfa71e8-kube-api-access-4cjzq\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:15:59.506612 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:15:59.506571    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/35686144-565a-4d97-b0c8-1fdd9dfa71e8-util\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:16:00.225430 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:00.225392    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn" event={"ID":"35686144-565a-4d97-b0c8-1fdd9dfa71e8","Type":"ContainerDied","Data":"70f87c18da221d5344028efa8e3098e0e8692d8be526e85a2faaed6b10dcf277"}
May 06 17:16:00.225430 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:00.225428    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="70f87c18da221d5344028efa8e3098e0e8692d8be526e85a2faaed6b10dcf277"
May 06 17:16:00.225430 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:00.225409    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/4c892a6d2d8a57acbc49427d3a7d24e253bab25be2c607ef405d2c4835sblcn"
May 06 17:16:04.965690 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.965655    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"]
May 06 17:16:04.966053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.965935    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35686144-565a-4d97-b0c8-1fdd9dfa71e8" containerName="extract"
May 06 17:16:04.966053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.965945    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="35686144-565a-4d97-b0c8-1fdd9dfa71e8" containerName="extract"
May 06 17:16:04.966053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.965961    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35686144-565a-4d97-b0c8-1fdd9dfa71e8" containerName="pull"
May 06 17:16:04.966053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.965967    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="35686144-565a-4d97-b0c8-1fdd9dfa71e8" containerName="pull"
May 06 17:16:04.966053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.965980    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="35686144-565a-4d97-b0c8-1fdd9dfa71e8" containerName="util"
May 06 17:16:04.966053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.965986    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="35686144-565a-4d97-b0c8-1fdd9dfa71e8" containerName="util"
May 06 17:16:04.966053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.966037    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="35686144-565a-4d97-b0c8-1fdd9dfa71e8" containerName="extract"
May 06 17:16:04.968859 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.968842    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:04.977283 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.977264    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-marketplace\"/\"default-dockercfg-tpsdd\""
May 06 17:16:04.980612 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.980594    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"openshift-service-ca.crt\""
May 06 17:16:04.990357 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.990333    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-marketplace\"/\"kube-root-ca.crt\""
May 06 17:16:04.991703 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:04.991671    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"]
May 06 17:16:05.052001 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:05.051969    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-bundle\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9\" (UID: \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:05.052229 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:05.052009    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-util\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9\" (UID: \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:05.052229 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:05.052038    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vnkbb\" (UniqueName: \"kubernetes.io/projected/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-kube-api-access-vnkbb\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9\" (UID: \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:05.153100 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:05.153062    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-bundle\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9\" (UID: \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:05.153100 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:05.153105    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-util\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9\" (UID: \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:05.153349 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:05.153230    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vnkbb\" (UniqueName: \"kubernetes.io/projected/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-kube-api-access-vnkbb\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9\" (UID: \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:05.153497 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:05.153477    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-util\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9\" (UID: \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:05.153568 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:05.153512    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-bundle\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9\" (UID: \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:05.188560 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:05.188526    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vnkbb\" (UniqueName: \"kubernetes.io/projected/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-kube-api-access-vnkbb\") pod \"7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9\" (UID: \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\") " pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:05.277596 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:05.277511    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:05.472377 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:05.472353    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"]
May 06 17:16:05.474990 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:16:05.474961    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa6948e4_ebfc_4d1a_916d_c952f54dfef6.slice/crio-9f58968b3063b57807d254d099bb1f07c27ebe0cd1adb30d3f7ade75f82192a2 WatchSource:0}: Error finding container 9f58968b3063b57807d254d099bb1f07c27ebe0cd1adb30d3f7ade75f82192a2: Status 404 returned error can't find the container with id 9f58968b3063b57807d254d099bb1f07c27ebe0cd1adb30d3f7ade75f82192a2
May 06 17:16:06.250896 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:06.250861    2560 generic.go:358] "Generic (PLEG): container finished" podID="fa6948e4-ebfc-4d1a-916d-c952f54dfef6" containerID="0607b70747622ac9bfd52dda366458138e3ea59e6619dd30b6df6c3fc7aa6866" exitCode=0
May 06 17:16:06.251370 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:06.250901    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9" event={"ID":"fa6948e4-ebfc-4d1a-916d-c952f54dfef6","Type":"ContainerDied","Data":"0607b70747622ac9bfd52dda366458138e3ea59e6619dd30b6df6c3fc7aa6866"}
May 06 17:16:06.251370 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:06.250925    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9" event={"ID":"fa6948e4-ebfc-4d1a-916d-c952f54dfef6","Type":"ContainerStarted","Data":"9f58968b3063b57807d254d099bb1f07c27ebe0cd1adb30d3f7ade75f82192a2"}
May 06 17:16:07.256128 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:07.256101    2560 generic.go:358] "Generic (PLEG): container finished" podID="fa6948e4-ebfc-4d1a-916d-c952f54dfef6" containerID="793ced77230749106fb6369e8d5b57cd12f3e689cec456179dde59ec8c7a0bc4" exitCode=0
May 06 17:16:07.256457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:07.256145    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9" event={"ID":"fa6948e4-ebfc-4d1a-916d-c952f54dfef6","Type":"ContainerDied","Data":"793ced77230749106fb6369e8d5b57cd12f3e689cec456179dde59ec8c7a0bc4"}
May 06 17:16:08.264003 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:08.263958    2560 generic.go:358] "Generic (PLEG): container finished" podID="fa6948e4-ebfc-4d1a-916d-c952f54dfef6" containerID="e01aaf3007ff47439fe9ba46c91316b0c5a966b6bc2db2e1c67e1a3e60926f27" exitCode=0
May 06 17:16:08.264381 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:08.264046    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9" event={"ID":"fa6948e4-ebfc-4d1a-916d-c952f54dfef6","Type":"ContainerDied","Data":"e01aaf3007ff47439fe9ba46c91316b0c5a966b6bc2db2e1c67e1a3e60926f27"}
May 06 17:16:09.387467 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:09.387446    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:09.493665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:09.493633    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-util\") pod \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\" (UID: \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\") "
May 06 17:16:09.493823 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:09.493694    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-bundle\") pod \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\" (UID: \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\") "
May 06 17:16:09.493823 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:09.493719    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-vnkbb\" (UniqueName: \"kubernetes.io/projected/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-kube-api-access-vnkbb\") pod \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\" (UID: \"fa6948e4-ebfc-4d1a-916d-c952f54dfef6\") "
May 06 17:16:09.494593 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:09.494563    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-bundle" (OuterVolumeSpecName: "bundle") pod "fa6948e4-ebfc-4d1a-916d-c952f54dfef6" (UID: "fa6948e4-ebfc-4d1a-916d-c952f54dfef6"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:16:09.495922 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:09.495898    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-kube-api-access-vnkbb" (OuterVolumeSpecName: "kube-api-access-vnkbb") pod "fa6948e4-ebfc-4d1a-916d-c952f54dfef6" (UID: "fa6948e4-ebfc-4d1a-916d-c952f54dfef6"). InnerVolumeSpecName "kube-api-access-vnkbb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:16:09.499201 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:09.499181    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-util" (OuterVolumeSpecName: "util") pod "fa6948e4-ebfc-4d1a-916d-c952f54dfef6" (UID: "fa6948e4-ebfc-4d1a-916d-c952f54dfef6"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:16:09.594514 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:09.594425    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-util\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:16:09.594514 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:09.594460    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-bundle\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:16:09.594514 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:09.594470    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-vnkbb\" (UniqueName: \"kubernetes.io/projected/fa6948e4-ebfc-4d1a-916d-c952f54dfef6-kube-api-access-vnkbb\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:16:10.272133 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:10.272096    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9" event={"ID":"fa6948e4-ebfc-4d1a-916d-c952f54dfef6","Type":"ContainerDied","Data":"9f58968b3063b57807d254d099bb1f07c27ebe0cd1adb30d3f7ade75f82192a2"}
May 06 17:16:10.272133 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:10.272130    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9f58968b3063b57807d254d099bb1f07c27ebe0cd1adb30d3f7ade75f82192a2"
May 06 17:16:10.272407 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:10.272181    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-marketplace/7797e60c2c2aaccf623f93d365c9b5c5cd3662e5f903c80e749ff805ebk7gt9"
May 06 17:16:38.080403 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.080358    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"]
May 06 17:16:38.080874 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.080791    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6948e4-ebfc-4d1a-916d-c952f54dfef6" containerName="util"
May 06 17:16:38.080874 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.080807    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6948e4-ebfc-4d1a-916d-c952f54dfef6" containerName="util"
May 06 17:16:38.080874 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.080825    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6948e4-ebfc-4d1a-916d-c952f54dfef6" containerName="pull"
May 06 17:16:38.080874 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.080833    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6948e4-ebfc-4d1a-916d-c952f54dfef6" containerName="pull"
May 06 17:16:38.080874 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.080858    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa6948e4-ebfc-4d1a-916d-c952f54dfef6" containerName="extract"
May 06 17:16:38.080874 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.080867    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa6948e4-ebfc-4d1a-916d-c952f54dfef6" containerName="extract"
May 06 17:16:38.081059 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.080936    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="fa6948e4-ebfc-4d1a-916d-c952f54dfef6" containerName="extract"
May 06 17:16:38.083619 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.083597    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.087489 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.087461    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-gw-ca-root-cert\""
May 06 17:16:38.087643 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.087620    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 06 17:16:38.088087 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.088050    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"data-science-gateway-data-science-gateway-class-dockercfg-xcs22\""
May 06 17:16:38.088198 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.088119    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 06 17:16:38.100929 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.100896    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"]
May 06 17:16:38.138017 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.137983    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.138017 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.138023    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.138249 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.138051    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-npmdv\" (UniqueName: \"kubernetes.io/projected/3171596d-a89d-4a55-bcfc-128408e618e0-kube-api-access-npmdv\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.138249 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.138191    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.138249 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.138231    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.138350 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.138282    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3171596d-a89d-4a55-bcfc-128408e618e0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.138350 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.138313    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3171596d-a89d-4a55-bcfc-128408e618e0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.138350 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.138344    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.138439 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.138373    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3171596d-a89d-4a55-bcfc-128408e618e0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.238818 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.238786    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.238818 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.238826    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.239053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.238952    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3171596d-a89d-4a55-bcfc-128408e618e0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.239053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.238997    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3171596d-a89d-4a55-bcfc-128408e618e0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.239053 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.239026    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.239254 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.239056    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3171596d-a89d-4a55-bcfc-128408e618e0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.239254 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.239096    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.239254 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.239196    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-credential-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.239393 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.239272    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-workload-certs\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.239452 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.239392    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.239501 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.239455    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-npmdv\" (UniqueName: \"kubernetes.io/projected/3171596d-a89d-4a55-bcfc-128408e618e0-kube-api-access-npmdv\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.239557 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.239510    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-workload-socket\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.239671 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.239650    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-istio-data\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.239806 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.239783    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/3171596d-a89d-4a55-bcfc-128408e618e0-istiod-ca-cert\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.241380 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.241354    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/3171596d-a89d-4a55-bcfc-128408e618e0-istio-envoy\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.241694 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.241676    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/3171596d-a89d-4a55-bcfc-128408e618e0-istio-podinfo\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.247919 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.247900    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/3171596d-a89d-4a55-bcfc-128408e618e0-istio-token\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.252686 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.252660    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-npmdv\" (UniqueName: \"kubernetes.io/projected/3171596d-a89d-4a55-bcfc-128408e618e0-kube-api-access-npmdv\") pod \"data-science-gateway-data-science-gateway-class-595b7776f8swjrs\" (UID: \"3171596d-a89d-4a55-bcfc-128408e618e0\") " pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.394091 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.394008    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:38.533681 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:38.533656    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"]
May 06 17:16:38.535466 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:16:38.535435    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3171596d_a89d_4a55_bcfc_128408e618e0.slice/crio-2408942d78db7a7b99f215fd755c6b5a5a8c971fc0938d38d4df7e58d0f44a63 WatchSource:0}: Error finding container 2408942d78db7a7b99f215fd755c6b5a5a8c971fc0938d38d4df7e58d0f44a63: Status 404 returned error can't find the container with id 2408942d78db7a7b99f215fd755c6b5a5a8c971fc0938d38d4df7e58d0f44a63
May 06 17:16:39.372371 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:39.372332    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs" event={"ID":"3171596d-a89d-4a55-bcfc-128408e618e0","Type":"ContainerStarted","Data":"2408942d78db7a7b99f215fd755c6b5a5a8c971fc0938d38d4df7e58d0f44a63"}
May 06 17:16:40.965921 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:40.965878    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
May 06 17:16:40.966187 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:40.965956    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
May 06 17:16:40.966187 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:40.965984    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
May 06 17:16:41.381362 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:41.381269    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs" event={"ID":"3171596d-a89d-4a55-bcfc-128408e618e0","Type":"ContainerStarted","Data":"773768c2dc5fc5592945b0d59118c121452f0c0e5ebcd6dd1bb452c08575504d"}
May 06 17:16:41.394312 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:41.394277    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:41.410060 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:41.410006    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs" podStartSLOduration=0.98168386 podStartE2EDuration="3.40999164s" podCreationTimestamp="2026-05-06 17:16:38 +0000 UTC" firstStartedPulling="2026-05-06 17:16:38.537354835 +0000 UTC m=+369.981177261" lastFinishedPulling="2026-05-06 17:16:40.965662616 +0000 UTC m=+372.409485041" observedRunningTime="2026-05-06 17:16:41.4068774 +0000 UTC m=+372.850699849" watchObservedRunningTime="2026-05-06 17:16:41.40999164 +0000 UTC m=+372.853814087"
May 06 17:16:42.398856 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:42.398823    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:43.388574 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:43.388542    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:43.389558 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:43.389538    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/data-science-gateway-data-science-gateway-class-595b7776f8swjrs"
May 06 17:16:53.613430 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:53.613397    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dblgc"]
May 06 17:16:53.615420 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:53.615404    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dblgc"
May 06 17:16:53.618603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:53.618574    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"openshift-service-ca.crt\""
May 06 17:16:53.619026 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:53.619006    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-catalog-dockercfg-qr5wv\""
May 06 17:16:53.619256 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:53.619232    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kube-root-ca.crt\""
May 06 17:16:53.633872 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:53.633839    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dblgc"]
May 06 17:16:53.668242 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:53.668204    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gjvp2\" (UniqueName: \"kubernetes.io/projected/4b3cd03a-0283-4375-87ce-c2e258c50eef-kube-api-access-gjvp2\") pod \"kuadrant-operator-catalog-dblgc\" (UID: \"4b3cd03a-0283-4375-87ce-c2e258c50eef\") " pod="kuadrant-system/kuadrant-operator-catalog-dblgc"
May 06 17:16:53.769665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:53.769625    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gjvp2\" (UniqueName: \"kubernetes.io/projected/4b3cd03a-0283-4375-87ce-c2e258c50eef-kube-api-access-gjvp2\") pod \"kuadrant-operator-catalog-dblgc\" (UID: \"4b3cd03a-0283-4375-87ce-c2e258c50eef\") " pod="kuadrant-system/kuadrant-operator-catalog-dblgc"
May 06 17:16:53.784531 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:53.784493    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gjvp2\" (UniqueName: \"kubernetes.io/projected/4b3cd03a-0283-4375-87ce-c2e258c50eef-kube-api-access-gjvp2\") pod \"kuadrant-operator-catalog-dblgc\" (UID: \"4b3cd03a-0283-4375-87ce-c2e258c50eef\") " pod="kuadrant-system/kuadrant-operator-catalog-dblgc"
May 06 17:16:53.927101 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:53.927052    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dblgc"
May 06 17:16:53.935248 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:53.935212    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dblgc"]
May 06 17:16:54.076648 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:54.076623    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dblgc"]
May 06 17:16:54.078712 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:16:54.078682    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4b3cd03a_0283_4375_87ce_c2e258c50eef.slice/crio-10169ce096f320120c3435341e35d02ccc5118ee5e03f61ba6bbf5c99ea7417e WatchSource:0}: Error finding container 10169ce096f320120c3435341e35d02ccc5118ee5e03f61ba6bbf5c99ea7417e: Status 404 returned error can't find the container with id 10169ce096f320120c3435341e35d02ccc5118ee5e03f61ba6bbf5c99ea7417e
May 06 17:16:54.143596 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:54.143564    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-29x4m"]
May 06 17:16:54.146395 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:54.146370    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-29x4m"
May 06 17:16:54.162513 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:54.162485    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-29x4m"]
May 06 17:16:54.172665 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:54.172640    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mk9dg\" (UniqueName: \"kubernetes.io/projected/2d7a32b4-3b9e-414f-bbac-a2492fa01c7c-kube-api-access-mk9dg\") pod \"kuadrant-operator-catalog-29x4m\" (UID: \"2d7a32b4-3b9e-414f-bbac-a2492fa01c7c\") " pod="kuadrant-system/kuadrant-operator-catalog-29x4m"
May 06 17:16:54.273373 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:54.273287    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mk9dg\" (UniqueName: \"kubernetes.io/projected/2d7a32b4-3b9e-414f-bbac-a2492fa01c7c-kube-api-access-mk9dg\") pod \"kuadrant-operator-catalog-29x4m\" (UID: \"2d7a32b4-3b9e-414f-bbac-a2492fa01c7c\") " pod="kuadrant-system/kuadrant-operator-catalog-29x4m"
May 06 17:16:54.286038 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:54.286011    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mk9dg\" (UniqueName: \"kubernetes.io/projected/2d7a32b4-3b9e-414f-bbac-a2492fa01c7c-kube-api-access-mk9dg\") pod \"kuadrant-operator-catalog-29x4m\" (UID: \"2d7a32b4-3b9e-414f-bbac-a2492fa01c7c\") " pod="kuadrant-system/kuadrant-operator-catalog-29x4m"
May 06 17:16:54.430292 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:54.430256    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dblgc" event={"ID":"4b3cd03a-0283-4375-87ce-c2e258c50eef","Type":"ContainerStarted","Data":"10169ce096f320120c3435341e35d02ccc5118ee5e03f61ba6bbf5c99ea7417e"}
May 06 17:16:54.455228 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:54.455201    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-29x4m"
May 06 17:16:54.596498 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:54.596318    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-29x4m"]
May 06 17:16:54.601039 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:16:54.601010    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2d7a32b4_3b9e_414f_bbac_a2492fa01c7c.slice/crio-23d1e46ebd4e9fb7eca3b2b2bf428a8fc780135a99b9f13c4382acd4de4c99a5 WatchSource:0}: Error finding container 23d1e46ebd4e9fb7eca3b2b2bf428a8fc780135a99b9f13c4382acd4de4c99a5: Status 404 returned error can't find the container with id 23d1e46ebd4e9fb7eca3b2b2bf428a8fc780135a99b9f13c4382acd4de4c99a5
May 06 17:16:55.436478 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:55.436437    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-29x4m" event={"ID":"2d7a32b4-3b9e-414f-bbac-a2492fa01c7c","Type":"ContainerStarted","Data":"23d1e46ebd4e9fb7eca3b2b2bf428a8fc780135a99b9f13c4382acd4de4c99a5"}
May 06 17:16:56.442338 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:56.442099    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dblgc" event={"ID":"4b3cd03a-0283-4375-87ce-c2e258c50eef","Type":"ContainerStarted","Data":"7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592"}
May 06 17:16:56.442338 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:56.442207    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-catalog-dblgc" podUID="4b3cd03a-0283-4375-87ce-c2e258c50eef" containerName="registry-server" containerID="cri-o://7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592" gracePeriod=2
May 06 17:16:56.443571 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:56.443536    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-29x4m" event={"ID":"2d7a32b4-3b9e-414f-bbac-a2492fa01c7c","Type":"ContainerStarted","Data":"4d597c5e8bdf6f5d7b2d7601bc6abcef1a21ef742dcef8103544f6f9fd8fc6b1"}
May 06 17:16:56.464812 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:56.464747    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-dblgc" podStartSLOduration=1.239732444 podStartE2EDuration="3.464729593s" podCreationTimestamp="2026-05-06 17:16:53 +0000 UTC" firstStartedPulling="2026-05-06 17:16:54.080008673 +0000 UTC m=+385.523831101" lastFinishedPulling="2026-05-06 17:16:56.305005823 +0000 UTC m=+387.748828250" observedRunningTime="2026-05-06 17:16:56.463946494 +0000 UTC m=+387.907768942" watchObservedRunningTime="2026-05-06 17:16:56.464729593 +0000 UTC m=+387.908552040"
May 06 17:16:56.492929 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:56.492869    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-catalog-29x4m" podStartSLOduration=0.787331297 podStartE2EDuration="2.492851121s" podCreationTimestamp="2026-05-06 17:16:54 +0000 UTC" firstStartedPulling="2026-05-06 17:16:54.602475556 +0000 UTC m=+386.046297981" lastFinishedPulling="2026-05-06 17:16:56.30799538 +0000 UTC m=+387.751817805" observedRunningTime="2026-05-06 17:16:56.492846441 +0000 UTC m=+387.936668890" watchObservedRunningTime="2026-05-06 17:16:56.492851121 +0000 UTC m=+387.936673569"
May 06 17:16:56.680637 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:56.680616    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dblgc_4b3cd03a-0283-4375-87ce-c2e258c50eef/registry-server/0.log"
May 06 17:16:56.680746 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:56.680675    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dblgc"
May 06 17:16:56.795407 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:56.795371    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gjvp2\" (UniqueName: \"kubernetes.io/projected/4b3cd03a-0283-4375-87ce-c2e258c50eef-kube-api-access-gjvp2\") pod \"4b3cd03a-0283-4375-87ce-c2e258c50eef\" (UID: \"4b3cd03a-0283-4375-87ce-c2e258c50eef\") "
May 06 17:16:56.797514 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:56.797492    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4b3cd03a-0283-4375-87ce-c2e258c50eef-kube-api-access-gjvp2" (OuterVolumeSpecName: "kube-api-access-gjvp2") pod "4b3cd03a-0283-4375-87ce-c2e258c50eef" (UID: "4b3cd03a-0283-4375-87ce-c2e258c50eef"). InnerVolumeSpecName "kube-api-access-gjvp2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:16:56.896117 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:56.896029    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gjvp2\" (UniqueName: \"kubernetes.io/projected/4b3cd03a-0283-4375-87ce-c2e258c50eef-kube-api-access-gjvp2\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:16:57.447940 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:57.447911    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-dblgc_4b3cd03a-0283-4375-87ce-c2e258c50eef/registry-server/0.log"
May 06 17:16:57.448342 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:57.447950    2560 generic.go:358] "Generic (PLEG): container finished" podID="4b3cd03a-0283-4375-87ce-c2e258c50eef" containerID="7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592" exitCode=1
May 06 17:16:57.448342 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:57.448043    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-catalog-dblgc"
May 06 17:16:57.448342 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:57.448045    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dblgc" event={"ID":"4b3cd03a-0283-4375-87ce-c2e258c50eef","Type":"ContainerDied","Data":"7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592"}
May 06 17:16:57.448342 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:57.448086    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-catalog-dblgc" event={"ID":"4b3cd03a-0283-4375-87ce-c2e258c50eef","Type":"ContainerDied","Data":"10169ce096f320120c3435341e35d02ccc5118ee5e03f61ba6bbf5c99ea7417e"}
May 06 17:16:57.448342 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:57.448101    2560 scope.go:117] "RemoveContainer" containerID="7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592"
May 06 17:16:57.456734 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:57.456718    2560 scope.go:117] "RemoveContainer" containerID="7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592"
May 06 17:16:57.456962 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:16:57.456942    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592\": container with ID starting with 7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592 not found: ID does not exist" containerID="7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592"
May 06 17:16:57.457025 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:57.456978    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592"} err="failed to get container status \"7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592\": rpc error: code = NotFound desc = could not find container \"7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592\": container with ID starting with 7d42f232b29ce1ea7da4b1dc9305a57bdb4d602b234adcd60150827bf4866592 not found: ID does not exist"
May 06 17:16:57.467946 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:57.467923    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dblgc"]
May 06 17:16:57.474240 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:57.474216    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-catalog-dblgc"]
May 06 17:16:59.121721 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:16:59.121688    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4b3cd03a-0283-4375-87ce-c2e258c50eef" path="/var/lib/kubelet/pods/4b3cd03a-0283-4375-87ce-c2e258c50eef/volumes"
May 06 17:17:04.456048 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:04.456008    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kuadrant-system/kuadrant-operator-catalog-29x4m"
May 06 17:17:04.456048 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:04.456055    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-catalog-29x4m"
May 06 17:17:04.477522 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:04.477490    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kuadrant-system/kuadrant-operator-catalog-29x4m"
May 06 17:17:05.496092 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:05.496063    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-catalog-29x4m"
May 06 17:17:09.172744 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.172701    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"]
May 06 17:17:09.173132 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.172997    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4b3cd03a-0283-4375-87ce-c2e258c50eef" containerName="registry-server"
May 06 17:17:09.173132 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.173008    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="4b3cd03a-0283-4375-87ce-c2e258c50eef" containerName="registry-server"
May 06 17:17:09.173132 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.173067    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="4b3cd03a-0283-4375-87ce-c2e258c50eef" containerName="registry-server"
May 06 17:17:09.177426 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.177410    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:09.179890 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.179867    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mswl4\""
May 06 17:17:09.188073 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.188049    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"]
May 06 17:17:09.301221 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.301182    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx\" (UID: \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:09.301373 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.301279    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qj26r\" (UniqueName: \"kubernetes.io/projected/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-kube-api-access-qj26r\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx\" (UID: \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:09.301373 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.301333    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx\" (UID: \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:09.402628 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.402589    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx\" (UID: \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:09.402783 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.402647    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qj26r\" (UniqueName: \"kubernetes.io/projected/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-kube-api-access-qj26r\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx\" (UID: \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:09.402783 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.402675    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx\" (UID: \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:09.402993 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.402974    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-util\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx\" (UID: \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:09.403036 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.403014    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-bundle\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx\" (UID: \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:09.413135 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.413111    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qj26r\" (UniqueName: \"kubernetes.io/projected/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-kube-api-access-qj26r\") pod \"0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx\" (UID: \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\") " pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:09.486708 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.486625    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:09.615022 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.614978    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"]
May 06 17:17:09.617784 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:17:09.617755    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfac5f175_4e7d_4e1b_a7d3_2cc92dcf0609.slice/crio-032fb325b5ad8ad1f6eb14c6469d63a54518684aafda588c7e13b9337873bd6d WatchSource:0}: Error finding container 032fb325b5ad8ad1f6eb14c6469d63a54518684aafda588c7e13b9337873bd6d: Status 404 returned error can't find the container with id 032fb325b5ad8ad1f6eb14c6469d63a54518684aafda588c7e13b9337873bd6d
May 06 17:17:09.810807 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.810733    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"]
May 06 17:17:09.814083 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.814068    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:09.827021 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.826993    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"]
May 06 17:17:09.907886 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.907857    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bjbds\" (UniqueName: \"kubernetes.io/projected/81e66fce-71ac-43ed-9f81-101513c9741e-kube-api-access-bjbds\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt\" (UID: \"81e66fce-71ac-43ed-9f81-101513c9741e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:09.908046 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.907894    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81e66fce-71ac-43ed-9f81-101513c9741e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt\" (UID: \"81e66fce-71ac-43ed-9f81-101513c9741e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:09.908046 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:09.907941    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81e66fce-71ac-43ed-9f81-101513c9741e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt\" (UID: \"81e66fce-71ac-43ed-9f81-101513c9741e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:10.009387 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.009345    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bjbds\" (UniqueName: \"kubernetes.io/projected/81e66fce-71ac-43ed-9f81-101513c9741e-kube-api-access-bjbds\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt\" (UID: \"81e66fce-71ac-43ed-9f81-101513c9741e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:10.009387 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.009392    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81e66fce-71ac-43ed-9f81-101513c9741e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt\" (UID: \"81e66fce-71ac-43ed-9f81-101513c9741e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:10.009579 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.009446    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81e66fce-71ac-43ed-9f81-101513c9741e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt\" (UID: \"81e66fce-71ac-43ed-9f81-101513c9741e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:10.009894 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.009874    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81e66fce-71ac-43ed-9f81-101513c9741e-bundle\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt\" (UID: \"81e66fce-71ac-43ed-9f81-101513c9741e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:10.009928 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.009884    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81e66fce-71ac-43ed-9f81-101513c9741e-util\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt\" (UID: \"81e66fce-71ac-43ed-9f81-101513c9741e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:10.018218 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.018189    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bjbds\" (UniqueName: \"kubernetes.io/projected/81e66fce-71ac-43ed-9f81-101513c9741e-kube-api-access-bjbds\") pod \"5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt\" (UID: \"81e66fce-71ac-43ed-9f81-101513c9741e\") " pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:10.123141 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.123060    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:10.170517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.170491    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"]
May 06 17:17:10.176423 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.176134    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:10.184521 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.184466    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"]
May 06 17:17:10.253335 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.253302    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"]
May 06 17:17:10.256277 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:17:10.256251    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod81e66fce_71ac_43ed_9f81_101513c9741e.slice/crio-9360f1dce3fa8b2539591739b3c0a44629361b5cb342c3ba739b453a1476f21b WatchSource:0}: Error finding container 9360f1dce3fa8b2539591739b3c0a44629361b5cb342c3ba739b453a1476f21b: Status 404 returned error can't find the container with id 9360f1dce3fa8b2539591739b3c0a44629361b5cb342c3ba739b453a1476f21b
May 06 17:17:10.311483 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.311458    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj\" (UID: \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:10.311590 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.311500    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj\" (UID: \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:10.311590 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.311539    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pxwwt\" (UniqueName: \"kubernetes.io/projected/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-kube-api-access-pxwwt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj\" (UID: \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:10.412843 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.412814    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj\" (UID: \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:10.412988 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.412853    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj\" (UID: \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:10.412988 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.412887    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pxwwt\" (UniqueName: \"kubernetes.io/projected/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-kube-api-access-pxwwt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj\" (UID: \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:10.413267 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.413244    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-util\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj\" (UID: \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:10.413331 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.413296    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-bundle\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj\" (UID: \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:10.423858 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.423831    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pxwwt\" (UniqueName: \"kubernetes.io/projected/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-kube-api-access-pxwwt\") pod \"19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj\" (UID: \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\") " pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:10.487223 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.487181    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:10.494994 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.492907    2560 generic.go:358] "Generic (PLEG): container finished" podID="81e66fce-71ac-43ed-9f81-101513c9741e" containerID="9ed2a8838edba2ddf2fe182d44f94faeda17ba5d4ddedc084779952588cf1d9b" exitCode=0
May 06 17:17:10.494994 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.493033    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt" event={"ID":"81e66fce-71ac-43ed-9f81-101513c9741e","Type":"ContainerDied","Data":"9ed2a8838edba2ddf2fe182d44f94faeda17ba5d4ddedc084779952588cf1d9b"}
May 06 17:17:10.494994 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.493063    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt" event={"ID":"81e66fce-71ac-43ed-9f81-101513c9741e","Type":"ContainerStarted","Data":"9360f1dce3fa8b2539591739b3c0a44629361b5cb342c3ba739b453a1476f21b"}
May 06 17:17:10.498076 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.498049    2560 generic.go:358] "Generic (PLEG): container finished" podID="fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" containerID="49b1a49e0602d82ca949fd96b3bc1789c03d5c72dec66fbca7e4d0725ddd2538" exitCode=0
May 06 17:17:10.498258 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.498143    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx" event={"ID":"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609","Type":"ContainerDied","Data":"49b1a49e0602d82ca949fd96b3bc1789c03d5c72dec66fbca7e4d0725ddd2538"}
May 06 17:17:10.498258 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.498235    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx" event={"ID":"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609","Type":"ContainerStarted","Data":"032fb325b5ad8ad1f6eb14c6469d63a54518684aafda588c7e13b9337873bd6d"}
May 06 17:17:10.608102 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.608073    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"]
May 06 17:17:10.613211 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.613192    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:10.624346 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.624323    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"]
May 06 17:17:10.638171 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:17:10.638110    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf268c192_5fa6_4fe2_82c0_16b8bf3f362d.slice/crio-a8f999779d91f9d30ea7655ef205dceb4d603570584666f0062bb3a8f3ecea2d WatchSource:0}: Error finding container a8f999779d91f9d30ea7655ef205dceb4d603570584666f0062bb3a8f3ecea2d: Status 404 returned error can't find the container with id a8f999779d91f9d30ea7655ef205dceb4d603570584666f0062bb3a8f3ecea2d
May 06 17:17:10.638401 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.638373    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"]
May 06 17:17:10.716047 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.716010    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/742300ba-658a-49b8-8af5-0cbac1d15924-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q\" (UID: \"742300ba-658a-49b8-8af5-0cbac1d15924\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:10.716201 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.716076    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/742300ba-658a-49b8-8af5-0cbac1d15924-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q\" (UID: \"742300ba-658a-49b8-8af5-0cbac1d15924\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:10.716201 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.716104    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s6gts\" (UniqueName: \"kubernetes.io/projected/742300ba-658a-49b8-8af5-0cbac1d15924-kube-api-access-s6gts\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q\" (UID: \"742300ba-658a-49b8-8af5-0cbac1d15924\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:10.817257 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.817223    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s6gts\" (UniqueName: \"kubernetes.io/projected/742300ba-658a-49b8-8af5-0cbac1d15924-kube-api-access-s6gts\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q\" (UID: \"742300ba-658a-49b8-8af5-0cbac1d15924\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:10.817430 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.817325    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/742300ba-658a-49b8-8af5-0cbac1d15924-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q\" (UID: \"742300ba-658a-49b8-8af5-0cbac1d15924\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:10.817430 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.817367    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/742300ba-658a-49b8-8af5-0cbac1d15924-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q\" (UID: \"742300ba-658a-49b8-8af5-0cbac1d15924\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:10.817774 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.817753    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/742300ba-658a-49b8-8af5-0cbac1d15924-util\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q\" (UID: \"742300ba-658a-49b8-8af5-0cbac1d15924\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:10.817811 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.817768    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/742300ba-658a-49b8-8af5-0cbac1d15924-bundle\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q\" (UID: \"742300ba-658a-49b8-8af5-0cbac1d15924\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:10.826481 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.826446    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s6gts\" (UniqueName: \"kubernetes.io/projected/742300ba-658a-49b8-8af5-0cbac1d15924-kube-api-access-s6gts\") pod \"9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q\" (UID: \"742300ba-658a-49b8-8af5-0cbac1d15924\") " pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:10.922675 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:10.922598    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:11.080493 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:11.080465    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"]
May 06 17:17:11.082653 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:17:11.082630    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod742300ba_658a_49b8_8af5_0cbac1d15924.slice/crio-e7d92a8e67083447bfae368ea814b9932fa9cfeb1a05b6c8af964cf2c6f49862 WatchSource:0}: Error finding container e7d92a8e67083447bfae368ea814b9932fa9cfeb1a05b6c8af964cf2c6f49862: Status 404 returned error can't find the container with id e7d92a8e67083447bfae368ea814b9932fa9cfeb1a05b6c8af964cf2c6f49862
May 06 17:17:11.506139 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:11.506110    2560 generic.go:358] "Generic (PLEG): container finished" podID="fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" containerID="26d73a584ccb1946a7e6a0fbf1ded4e684b2ba9cb245a9916ffeed83771d43c3" exitCode=0
May 06 17:17:11.506492 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:11.506191    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx" event={"ID":"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609","Type":"ContainerDied","Data":"26d73a584ccb1946a7e6a0fbf1ded4e684b2ba9cb245a9916ffeed83771d43c3"}
May 06 17:17:11.507669 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:11.507648    2560 generic.go:358] "Generic (PLEG): container finished" podID="742300ba-658a-49b8-8af5-0cbac1d15924" containerID="e45c042c4af9140749be7fde7f3591fbedfe3d00a4b0d0b47f83d3b8e4b1baf6" exitCode=0
May 06 17:17:11.507765 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:11.507723    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q" event={"ID":"742300ba-658a-49b8-8af5-0cbac1d15924","Type":"ContainerDied","Data":"e45c042c4af9140749be7fde7f3591fbedfe3d00a4b0d0b47f83d3b8e4b1baf6"}
May 06 17:17:11.507765 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:11.507747    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q" event={"ID":"742300ba-658a-49b8-8af5-0cbac1d15924","Type":"ContainerStarted","Data":"e7d92a8e67083447bfae368ea814b9932fa9cfeb1a05b6c8af964cf2c6f49862"}
May 06 17:17:11.509049 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:11.509028    2560 generic.go:358] "Generic (PLEG): container finished" podID="f268c192-5fa6-4fe2-82c0-16b8bf3f362d" containerID="581b3bee59e34403edb6ffb128b51af873d5494a1ad4fcf580381933d73157ca" exitCode=0
May 06 17:17:11.509144 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:11.509094    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj" event={"ID":"f268c192-5fa6-4fe2-82c0-16b8bf3f362d","Type":"ContainerDied","Data":"581b3bee59e34403edb6ffb128b51af873d5494a1ad4fcf580381933d73157ca"}
May 06 17:17:11.509144 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:11.509125    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj" event={"ID":"f268c192-5fa6-4fe2-82c0-16b8bf3f362d","Type":"ContainerStarted","Data":"a8f999779d91f9d30ea7655ef205dceb4d603570584666f0062bb3a8f3ecea2d"}
May 06 17:17:11.510635 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:11.510616    2560 generic.go:358] "Generic (PLEG): container finished" podID="81e66fce-71ac-43ed-9f81-101513c9741e" containerID="2421d4491886c9ca00d775aa46d2629bfd99ae42c34324d2640c2c99c2562bf4" exitCode=0
May 06 17:17:11.510723 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:11.510680    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt" event={"ID":"81e66fce-71ac-43ed-9f81-101513c9741e","Type":"ContainerDied","Data":"2421d4491886c9ca00d775aa46d2629bfd99ae42c34324d2640c2c99c2562bf4"}
May 06 17:17:12.516554 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:12.516519    2560 generic.go:358] "Generic (PLEG): container finished" podID="f268c192-5fa6-4fe2-82c0-16b8bf3f362d" containerID="ac514c3a8a93055fda58fd98d15918374f8761837a04323ace8845d73d8c0511" exitCode=0
May 06 17:17:12.516975 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:12.516611    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj" event={"ID":"f268c192-5fa6-4fe2-82c0-16b8bf3f362d","Type":"ContainerDied","Data":"ac514c3a8a93055fda58fd98d15918374f8761837a04323ace8845d73d8c0511"}
May 06 17:17:12.518421 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:12.518392    2560 generic.go:358] "Generic (PLEG): container finished" podID="81e66fce-71ac-43ed-9f81-101513c9741e" containerID="ec332435ef61a7b8ef4247a5bbdd441f77d3792e05228f55d5e2e957be5d5bec" exitCode=0
May 06 17:17:12.518504 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:12.518473    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt" event={"ID":"81e66fce-71ac-43ed-9f81-101513c9741e","Type":"ContainerDied","Data":"ec332435ef61a7b8ef4247a5bbdd441f77d3792e05228f55d5e2e957be5d5bec"}
May 06 17:17:12.520272 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:12.520254    2560 generic.go:358] "Generic (PLEG): container finished" podID="fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" containerID="d39deb08175f855c4b0a03daf369d96a6a64fd8fc336e36203b8170f7944958e" exitCode=0
May 06 17:17:12.520359 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:12.520330    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx" event={"ID":"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609","Type":"ContainerDied","Data":"d39deb08175f855c4b0a03daf369d96a6a64fd8fc336e36203b8170f7944958e"}
May 06 17:17:13.525125 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.525092    2560 generic.go:358] "Generic (PLEG): container finished" podID="742300ba-658a-49b8-8af5-0cbac1d15924" containerID="484030f94b234f9c8c442edfee153515c6d533aec48eb9285ffd91a23a45d10a" exitCode=0
May 06 17:17:13.525585 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.525179    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q" event={"ID":"742300ba-658a-49b8-8af5-0cbac1d15924","Type":"ContainerDied","Data":"484030f94b234f9c8c442edfee153515c6d533aec48eb9285ffd91a23a45d10a"}
May 06 17:17:13.527177 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.527133    2560 generic.go:358] "Generic (PLEG): container finished" podID="f268c192-5fa6-4fe2-82c0-16b8bf3f362d" containerID="ce94edf9f530827f9b962a1b41b771393f6136f7a0f303f2bc7e152b8d4a74f4" exitCode=0
May 06 17:17:13.527243 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.527195    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj" event={"ID":"f268c192-5fa6-4fe2-82c0-16b8bf3f362d","Type":"ContainerDied","Data":"ce94edf9f530827f9b962a1b41b771393f6136f7a0f303f2bc7e152b8d4a74f4"}
May 06 17:17:13.662788 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.662762    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:13.743661 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.743633    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81e66fce-71ac-43ed-9f81-101513c9741e-bundle\") pod \"81e66fce-71ac-43ed-9f81-101513c9741e\" (UID: \"81e66fce-71ac-43ed-9f81-101513c9741e\") "
May 06 17:17:13.743804 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.743699    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-bjbds\" (UniqueName: \"kubernetes.io/projected/81e66fce-71ac-43ed-9f81-101513c9741e-kube-api-access-bjbds\") pod \"81e66fce-71ac-43ed-9f81-101513c9741e\" (UID: \"81e66fce-71ac-43ed-9f81-101513c9741e\") "
May 06 17:17:13.743804 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.743775    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81e66fce-71ac-43ed-9f81-101513c9741e-util\") pod \"81e66fce-71ac-43ed-9f81-101513c9741e\" (UID: \"81e66fce-71ac-43ed-9f81-101513c9741e\") "
May 06 17:17:13.744335 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.744306    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81e66fce-71ac-43ed-9f81-101513c9741e-bundle" (OuterVolumeSpecName: "bundle") pod "81e66fce-71ac-43ed-9f81-101513c9741e" (UID: "81e66fce-71ac-43ed-9f81-101513c9741e"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:17:13.745789 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.745768    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/81e66fce-71ac-43ed-9f81-101513c9741e-kube-api-access-bjbds" (OuterVolumeSpecName: "kube-api-access-bjbds") pod "81e66fce-71ac-43ed-9f81-101513c9741e" (UID: "81e66fce-71ac-43ed-9f81-101513c9741e"). InnerVolumeSpecName "kube-api-access-bjbds". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:17:13.749279 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.749258    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:13.749553 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.749533    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/81e66fce-71ac-43ed-9f81-101513c9741e-util" (OuterVolumeSpecName: "util") pod "81e66fce-71ac-43ed-9f81-101513c9741e" (UID: "81e66fce-71ac-43ed-9f81-101513c9741e"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:17:13.844325 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.844241    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-bundle\") pod \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\" (UID: \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\") "
May 06 17:17:13.844325 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.844289    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qj26r\" (UniqueName: \"kubernetes.io/projected/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-kube-api-access-qj26r\") pod \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\" (UID: \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\") "
May 06 17:17:13.844523 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.844338    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-util\") pod \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\" (UID: \"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609\") "
May 06 17:17:13.844578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.844521    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-bjbds\" (UniqueName: \"kubernetes.io/projected/81e66fce-71ac-43ed-9f81-101513c9741e-kube-api-access-bjbds\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:17:13.844578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.844534    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/81e66fce-71ac-43ed-9f81-101513c9741e-util\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:17:13.844578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.844544    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/81e66fce-71ac-43ed-9f81-101513c9741e-bundle\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:17:13.844784 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.844759    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-bundle" (OuterVolumeSpecName: "bundle") pod "fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" (UID: "fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:17:13.846593 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.846559    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-kube-api-access-qj26r" (OuterVolumeSpecName: "kube-api-access-qj26r") pod "fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" (UID: "fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609"). InnerVolumeSpecName "kube-api-access-qj26r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:17:13.849828 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.849789    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-util" (OuterVolumeSpecName: "util") pod "fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" (UID: "fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:17:13.945786 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.945750    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-bundle\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:17:13.945786 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.945779    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qj26r\" (UniqueName: \"kubernetes.io/projected/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-kube-api-access-qj26r\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:17:13.945786 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:13.945790    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609-util\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:17:14.532477 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.532436    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx"
May 06 17:17:14.532477 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.532452    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx" event={"ID":"fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609","Type":"ContainerDied","Data":"032fb325b5ad8ad1f6eb14c6469d63a54518684aafda588c7e13b9337873bd6d"}
May 06 17:17:14.533041 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.532487    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="032fb325b5ad8ad1f6eb14c6469d63a54518684aafda588c7e13b9337873bd6d"
May 06 17:17:14.534342 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.534315    2560 generic.go:358] "Generic (PLEG): container finished" podID="742300ba-658a-49b8-8af5-0cbac1d15924" containerID="6be48e12381afed38822becf190df44339eb2285e1f1c5db81e8beebc09c6ecf" exitCode=0
May 06 17:17:14.534486 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.534413    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q" event={"ID":"742300ba-658a-49b8-8af5-0cbac1d15924","Type":"ContainerDied","Data":"6be48e12381afed38822becf190df44339eb2285e1f1c5db81e8beebc09c6ecf"}
May 06 17:17:14.536013 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.535987    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt" event={"ID":"81e66fce-71ac-43ed-9f81-101513c9741e","Type":"ContainerDied","Data":"9360f1dce3fa8b2539591739b3c0a44629361b5cb342c3ba739b453a1476f21b"}
May 06 17:17:14.536013 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.536014    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="9360f1dce3fa8b2539591739b3c0a44629361b5cb342c3ba739b453a1476f21b"
May 06 17:17:14.536199 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.536025    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt"
May 06 17:17:14.661626 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.661604    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:14.752613 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.752585    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pxwwt\" (UniqueName: \"kubernetes.io/projected/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-kube-api-access-pxwwt\") pod \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\" (UID: \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\") "
May 06 17:17:14.752778 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.752628    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-bundle\") pod \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\" (UID: \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\") "
May 06 17:17:14.752778 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.752700    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-util\") pod \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\" (UID: \"f268c192-5fa6-4fe2-82c0-16b8bf3f362d\") "
May 06 17:17:14.753170 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.753114    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-bundle" (OuterVolumeSpecName: "bundle") pod "f268c192-5fa6-4fe2-82c0-16b8bf3f362d" (UID: "f268c192-5fa6-4fe2-82c0-16b8bf3f362d"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:17:14.754758 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.754729    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-kube-api-access-pxwwt" (OuterVolumeSpecName: "kube-api-access-pxwwt") pod "f268c192-5fa6-4fe2-82c0-16b8bf3f362d" (UID: "f268c192-5fa6-4fe2-82c0-16b8bf3f362d"). InnerVolumeSpecName "kube-api-access-pxwwt". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:17:14.757930 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.757898    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-util" (OuterVolumeSpecName: "util") pod "f268c192-5fa6-4fe2-82c0-16b8bf3f362d" (UID: "f268c192-5fa6-4fe2-82c0-16b8bf3f362d"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:17:14.853329 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.853248    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-util\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:17:14.853329 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.853277    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pxwwt\" (UniqueName: \"kubernetes.io/projected/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-kube-api-access-pxwwt\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:17:14.853329 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:14.853289    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/f268c192-5fa6-4fe2-82c0-16b8bf3f362d-bundle\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:17:15.540488 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.540458    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj"
May 06 17:17:15.540488 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.540456    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj" event={"ID":"f268c192-5fa6-4fe2-82c0-16b8bf3f362d","Type":"ContainerDied","Data":"a8f999779d91f9d30ea7655ef205dceb4d603570584666f0062bb3a8f3ecea2d"}
May 06 17:17:15.540902 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.540503    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="a8f999779d91f9d30ea7655ef205dceb4d603570584666f0062bb3a8f3ecea2d"
May 06 17:17:15.662739 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.662718    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:15.762007 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.761969    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/742300ba-658a-49b8-8af5-0cbac1d15924-bundle\") pod \"742300ba-658a-49b8-8af5-0cbac1d15924\" (UID: \"742300ba-658a-49b8-8af5-0cbac1d15924\") "
May 06 17:17:15.762197 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.762073    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s6gts\" (UniqueName: \"kubernetes.io/projected/742300ba-658a-49b8-8af5-0cbac1d15924-kube-api-access-s6gts\") pod \"742300ba-658a-49b8-8af5-0cbac1d15924\" (UID: \"742300ba-658a-49b8-8af5-0cbac1d15924\") "
May 06 17:17:15.762197 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.762134    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/742300ba-658a-49b8-8af5-0cbac1d15924-util\") pod \"742300ba-658a-49b8-8af5-0cbac1d15924\" (UID: \"742300ba-658a-49b8-8af5-0cbac1d15924\") "
May 06 17:17:15.762873 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.762848    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/742300ba-658a-49b8-8af5-0cbac1d15924-bundle" (OuterVolumeSpecName: "bundle") pod "742300ba-658a-49b8-8af5-0cbac1d15924" (UID: "742300ba-658a-49b8-8af5-0cbac1d15924"). InnerVolumeSpecName "bundle". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:17:15.764269 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.764242    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/742300ba-658a-49b8-8af5-0cbac1d15924-kube-api-access-s6gts" (OuterVolumeSpecName: "kube-api-access-s6gts") pod "742300ba-658a-49b8-8af5-0cbac1d15924" (UID: "742300ba-658a-49b8-8af5-0cbac1d15924"). InnerVolumeSpecName "kube-api-access-s6gts". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:17:15.767333 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.767308    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/742300ba-658a-49b8-8af5-0cbac1d15924-util" (OuterVolumeSpecName: "util") pod "742300ba-658a-49b8-8af5-0cbac1d15924" (UID: "742300ba-658a-49b8-8af5-0cbac1d15924"). InnerVolumeSpecName "util". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:17:15.863807 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.863714    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s6gts\" (UniqueName: \"kubernetes.io/projected/742300ba-658a-49b8-8af5-0cbac1d15924-kube-api-access-s6gts\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:17:15.863807 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.863752    2560 reconciler_common.go:299] "Volume detached for volume \"util\" (UniqueName: \"kubernetes.io/empty-dir/742300ba-658a-49b8-8af5-0cbac1d15924-util\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:17:15.863807 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:15.863765    2560 reconciler_common.go:299] "Volume detached for volume \"bundle\" (UniqueName: \"kubernetes.io/empty-dir/742300ba-658a-49b8-8af5-0cbac1d15924-bundle\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:17:16.545246 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:16.545215    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q" event={"ID":"742300ba-658a-49b8-8af5-0cbac1d15924","Type":"ContainerDied","Data":"e7d92a8e67083447bfae368ea814b9932fa9cfeb1a05b6c8af964cf2c6f49862"}
May 06 17:17:16.545246 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:16.545252    2560 pod_container_deletor.go:80] "Container not found in pod's containers" containerID="e7d92a8e67083447bfae368ea814b9932fa9cfeb1a05b6c8af964cf2c6f49862"
May 06 17:17:16.545667 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:16.545227    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q"
May 06 17:17:26.921749 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.921717    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-bbncm"]
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922034    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81e66fce-71ac-43ed-9f81-101513c9741e" containerName="util"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922045    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="81e66fce-71ac-43ed-9f81-101513c9741e" containerName="util"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922059    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f268c192-5fa6-4fe2-82c0-16b8bf3f362d" containerName="util"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922066    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f268c192-5fa6-4fe2-82c0-16b8bf3f362d" containerName="util"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922074    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81e66fce-71ac-43ed-9f81-101513c9741e" containerName="pull"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922079    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="81e66fce-71ac-43ed-9f81-101513c9741e" containerName="pull"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922087    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="742300ba-658a-49b8-8af5-0cbac1d15924" containerName="pull"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922092    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="742300ba-658a-49b8-8af5-0cbac1d15924" containerName="pull"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922098    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f268c192-5fa6-4fe2-82c0-16b8bf3f362d" containerName="pull"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922103    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f268c192-5fa6-4fe2-82c0-16b8bf3f362d" containerName="pull"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922109    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="f268c192-5fa6-4fe2-82c0-16b8bf3f362d" containerName="extract"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922114    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="f268c192-5fa6-4fe2-82c0-16b8bf3f362d" containerName="extract"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922120    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="742300ba-658a-49b8-8af5-0cbac1d15924" containerName="util"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922124    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="742300ba-658a-49b8-8af5-0cbac1d15924" containerName="util"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922131    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="81e66fce-71ac-43ed-9f81-101513c9741e" containerName="extract"
May 06 17:17:26.922129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922136    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="81e66fce-71ac-43ed-9f81-101513c9741e" containerName="extract"
May 06 17:17:26.922603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922144    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" containerName="extract"
May 06 17:17:26.922603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922163    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" containerName="extract"
May 06 17:17:26.922603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922173    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" containerName="util"
May 06 17:17:26.922603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922178    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" containerName="util"
May 06 17:17:26.922603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922183    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" containerName="pull"
May 06 17:17:26.922603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922188    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" containerName="pull"
May 06 17:17:26.922603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922192    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="742300ba-658a-49b8-8af5-0cbac1d15924" containerName="extract"
May 06 17:17:26.922603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922197    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="742300ba-658a-49b8-8af5-0cbac1d15924" containerName="extract"
May 06 17:17:26.922603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922249    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="f268c192-5fa6-4fe2-82c0-16b8bf3f362d" containerName="extract"
May 06 17:17:26.922603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922256    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="742300ba-658a-49b8-8af5-0cbac1d15924" containerName="extract"
May 06 17:17:26.922603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922262    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609" containerName="extract"
May 06 17:17:26.922603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.922267    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="81e66fce-71ac-43ed-9f81-101513c9741e" containerName="extract"
May 06 17:17:26.926464 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.926445    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-bbncm"
May 06 17:17:26.929598 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.929578    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-operator-dockercfg-kxmm7\""
May 06 17:17:26.952933 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:26.952902    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-bbncm"]
May 06 17:17:27.061673 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:27.061639    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pgj2f\" (UniqueName: \"kubernetes.io/projected/e7f5efcb-bdc0-40d3-a2c0-dad2ad5133c7-kube-api-access-pgj2f\") pod \"authorino-operator-657f44b778-bbncm\" (UID: \"e7f5efcb-bdc0-40d3-a2c0-dad2ad5133c7\") " pod="kuadrant-system/authorino-operator-657f44b778-bbncm"
May 06 17:17:27.162924 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:27.162893    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pgj2f\" (UniqueName: \"kubernetes.io/projected/e7f5efcb-bdc0-40d3-a2c0-dad2ad5133c7-kube-api-access-pgj2f\") pod \"authorino-operator-657f44b778-bbncm\" (UID: \"e7f5efcb-bdc0-40d3-a2c0-dad2ad5133c7\") " pod="kuadrant-system/authorino-operator-657f44b778-bbncm"
May 06 17:17:27.206489 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:27.206403    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pgj2f\" (UniqueName: \"kubernetes.io/projected/e7f5efcb-bdc0-40d3-a2c0-dad2ad5133c7-kube-api-access-pgj2f\") pod \"authorino-operator-657f44b778-bbncm\" (UID: \"e7f5efcb-bdc0-40d3-a2c0-dad2ad5133c7\") " pod="kuadrant-system/authorino-operator-657f44b778-bbncm"
May 06 17:17:27.236177 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:27.236131    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-operator-657f44b778-bbncm"
May 06 17:17:27.389387 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:27.389355    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-operator-657f44b778-bbncm"]
May 06 17:17:27.390991 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:17:27.390964    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode7f5efcb_bdc0_40d3_a2c0_dad2ad5133c7.slice/crio-24292d3178715f95833c5b5e04fa443ad6567cd3d664d18e8f4fabab8e075106 WatchSource:0}: Error finding container 24292d3178715f95833c5b5e04fa443ad6567cd3d664d18e8f4fabab8e075106: Status 404 returned error can't find the container with id 24292d3178715f95833c5b5e04fa443ad6567cd3d664d18e8f4fabab8e075106
May 06 17:17:27.583189 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:27.583090    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-bbncm" event={"ID":"e7f5efcb-bdc0-40d3-a2c0-dad2ad5133c7","Type":"ContainerStarted","Data":"24292d3178715f95833c5b5e04fa443ad6567cd3d664d18e8f4fabab8e075106"}
May 06 17:17:30.597587 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:30.597545    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-operator-657f44b778-bbncm" event={"ID":"e7f5efcb-bdc0-40d3-a2c0-dad2ad5133c7","Type":"ContainerStarted","Data":"068549f51284fd9c4153259e56f52f35a3f3fd895ddf90422a313c950c8c062a"}
May 06 17:17:30.598043 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:30.597623    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/authorino-operator-657f44b778-bbncm"
May 06 17:17:30.628815 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:30.628766    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-operator-657f44b778-bbncm" podStartSLOduration=2.428606557 podStartE2EDuration="4.628751122s" podCreationTimestamp="2026-05-06 17:17:26 +0000 UTC" firstStartedPulling="2026-05-06 17:17:27.393369142 +0000 UTC m=+418.837191567" lastFinishedPulling="2026-05-06 17:17:29.593513707 +0000 UTC m=+421.037336132" observedRunningTime="2026-05-06 17:17:30.628130134 +0000 UTC m=+422.071952580" watchObservedRunningTime="2026-05-06 17:17:30.628751122 +0000 UTC m=+422.072573589"
May 06 17:17:30.975887 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:30.975847    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"]
May 06 17:17:30.979485 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:30.979468    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"
May 06 17:17:30.982252 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:30.982230    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"kuadrant-operator-controller-manager-dockercfg-tx98p\""
May 06 17:17:30.994227 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:30.994204    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lx2vn\" (UniqueName: \"kubernetes.io/projected/b70d58c0-5daa-4064-99fc-195823bd93b4-kube-api-access-lx2vn\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-mckvd\" (UID: \"b70d58c0-5daa-4064-99fc-195823bd93b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"
May 06 17:17:30.994330 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:30.994269    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70d58c0-5daa-4064-99fc-195823bd93b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-mckvd\" (UID: \"b70d58c0-5daa-4064-99fc-195823bd93b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"
May 06 17:17:30.994401 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:30.994383    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"]
May 06 17:17:31.095651 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:31.095618    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lx2vn\" (UniqueName: \"kubernetes.io/projected/b70d58c0-5daa-4064-99fc-195823bd93b4-kube-api-access-lx2vn\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-mckvd\" (UID: \"b70d58c0-5daa-4064-99fc-195823bd93b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"
May 06 17:17:31.095823 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:31.095677    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70d58c0-5daa-4064-99fc-195823bd93b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-mckvd\" (UID: \"b70d58c0-5daa-4064-99fc-195823bd93b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"
May 06 17:17:31.096063 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:31.096041    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70d58c0-5daa-4064-99fc-195823bd93b4-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-mckvd\" (UID: \"b70d58c0-5daa-4064-99fc-195823bd93b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"
May 06 17:17:31.106104 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:31.106074    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lx2vn\" (UniqueName: \"kubernetes.io/projected/b70d58c0-5daa-4064-99fc-195823bd93b4-kube-api-access-lx2vn\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-mckvd\" (UID: \"b70d58c0-5daa-4064-99fc-195823bd93b4\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"
May 06 17:17:31.289404 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:31.289322    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"
May 06 17:17:31.456234 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:31.456204    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"]
May 06 17:17:31.458548 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:17:31.458518    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb70d58c0_5daa_4064_99fc_195823bd93b4.slice/crio-18c92ab0e214cb782c122a50fa81057f4d09e0d9e2bf726257e984a9b813f402 WatchSource:0}: Error finding container 18c92ab0e214cb782c122a50fa81057f4d09e0d9e2bf726257e984a9b813f402: Status 404 returned error can't find the container with id 18c92ab0e214cb782c122a50fa81057f4d09e0d9e2bf726257e984a9b813f402
May 06 17:17:31.602417 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:31.602337    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd" event={"ID":"b70d58c0-5daa-4064-99fc-195823bd93b4","Type":"ContainerStarted","Data":"18c92ab0e214cb782c122a50fa81057f4d09e0d9e2bf726257e984a9b813f402"}
May 06 17:17:37.633485 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:37.633444    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd" event={"ID":"b70d58c0-5daa-4064-99fc-195823bd93b4","Type":"ContainerStarted","Data":"c247e525e31245fa7c5ddf78e1b2dc4da4ccfdce6c96385890924bb7ac3c4c93"}
May 06 17:17:37.633846 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:37.633556    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"
May 06 17:17:37.663081 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:37.663031    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd" podStartSLOduration=2.258986996 podStartE2EDuration="7.663016357s" podCreationTimestamp="2026-05-06 17:17:30 +0000 UTC" firstStartedPulling="2026-05-06 17:17:31.460958978 +0000 UTC m=+422.904781403" lastFinishedPulling="2026-05-06 17:17:36.864988339 +0000 UTC m=+428.308810764" observedRunningTime="2026-05-06 17:17:37.662271628 +0000 UTC m=+429.106094075" watchObservedRunningTime="2026-05-06 17:17:37.663016357 +0000 UTC m=+429.106838803"
May 06 17:17:39.775720 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.775683    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"]
May 06 17:17:39.779063 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.779043    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"
May 06 17:17:39.783018 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.783000    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"plugin-serving-cert\""
May 06 17:17:39.783105 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.783004    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"kuadrant-console-nginx-conf\""
May 06 17:17:39.783105 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.783056    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"default-dockercfg-mswl4\""
May 06 17:17:39.796300 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.796272    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"]
May 06 17:17:39.874217 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.874180    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-v6ks2\" (UniqueName: \"kubernetes.io/projected/80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f-kube-api-access-v6ks2\") pod \"kuadrant-console-plugin-6cb54b5c86-gtpm4\" (UID: \"80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"
May 06 17:17:39.874418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.874243    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-gtpm4\" (UID: \"80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"
May 06 17:17:39.874418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.874276    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-gtpm4\" (UID: \"80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"
May 06 17:17:39.975196 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.975147    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-gtpm4\" (UID: \"80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"
May 06 17:17:39.975196 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.975201    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-gtpm4\" (UID: \"80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"
May 06 17:17:39.975418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.975268    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-v6ks2\" (UniqueName: \"kubernetes.io/projected/80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f-kube-api-access-v6ks2\") pod \"kuadrant-console-plugin-6cb54b5c86-gtpm4\" (UID: \"80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"
May 06 17:17:39.975870 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.975849    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f-nginx-conf\") pod \"kuadrant-console-plugin-6cb54b5c86-gtpm4\" (UID: \"80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"
May 06 17:17:39.977583 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.977563    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"plugin-serving-cert\" (UniqueName: \"kubernetes.io/secret/80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f-plugin-serving-cert\") pod \"kuadrant-console-plugin-6cb54b5c86-gtpm4\" (UID: \"80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"
May 06 17:17:39.985539 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:39.985512    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-v6ks2\" (UniqueName: \"kubernetes.io/projected/80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f-kube-api-access-v6ks2\") pod \"kuadrant-console-plugin-6cb54b5c86-gtpm4\" (UID: \"80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f\") " pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"
May 06 17:17:40.088701 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:40.088609    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"
May 06 17:17:40.221034 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:40.221009    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4"]
May 06 17:17:40.223189 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:17:40.223121    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod80ea41c0_c4c0_4fc7_9fc1_2d510a71f14f.slice/crio-6604ef7b3cb69a41f792a2e744ba0ad65dda33bf2b111da670d6c51d6a5511a7 WatchSource:0}: Error finding container 6604ef7b3cb69a41f792a2e744ba0ad65dda33bf2b111da670d6c51d6a5511a7: Status 404 returned error can't find the container with id 6604ef7b3cb69a41f792a2e744ba0ad65dda33bf2b111da670d6c51d6a5511a7
May 06 17:17:40.646081 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:40.646040    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4" event={"ID":"80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f","Type":"ContainerStarted","Data":"6604ef7b3cb69a41f792a2e744ba0ad65dda33bf2b111da670d6c51d6a5511a7"}
May 06 17:17:41.605736 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:41.605694    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/authorino-operator-657f44b778-bbncm"
May 06 17:17:48.640008 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:48.639924    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"
May 06 17:17:50.605002 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.604966    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"]
May 06 17:17:50.605474 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.605291    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd" podUID="b70d58c0-5daa-4064-99fc-195823bd93b4" containerName="manager" containerID="cri-o://c247e525e31245fa7c5ddf78e1b2dc4da4ccfdce6c96385890924bb7ac3c4c93" gracePeriod=2
May 06 17:17:50.617092 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.617038    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"]
May 06 17:17:50.633525 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.633492    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"]
May 06 17:17:50.634052 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.634010    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b70d58c0-5daa-4064-99fc-195823bd93b4" containerName="manager"
May 06 17:17:50.634052 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.634033    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b70d58c0-5daa-4064-99fc-195823bd93b4" containerName="manager"
May 06 17:17:50.634316 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.634115    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="b70d58c0-5daa-4064-99fc-195823bd93b4" containerName="manager"
May 06 17:17:50.637918 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.637896    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"
May 06 17:17:50.657574 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.657536    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"]
May 06 17:17:50.686913 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.686872    2560 status_manager.go:895] "Failed to get status for pod" podUID="b70d58c0-5daa-4064-99fc-195823bd93b4" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mckvd\" is forbidden: User \"system:node:ip-10-0-130-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-186.ec2.internal' and this object"
May 06 17:17:50.781493 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.781455    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2edbf134-a1ed-436a-9404-4e192964e49a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-m2gdn\" (UID: \"2edbf134-a1ed-436a-9404-4e192964e49a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"
May 06 17:17:50.781663 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.781520    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-456dc\" (UniqueName: \"kubernetes.io/projected/2edbf134-a1ed-436a-9404-4e192964e49a-kube-api-access-456dc\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-m2gdn\" (UID: \"2edbf134-a1ed-436a-9404-4e192964e49a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"
May 06 17:17:50.882959 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.882838    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2edbf134-a1ed-436a-9404-4e192964e49a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-m2gdn\" (UID: \"2edbf134-a1ed-436a-9404-4e192964e49a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"
May 06 17:17:50.882959 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.882911    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-456dc\" (UniqueName: \"kubernetes.io/projected/2edbf134-a1ed-436a-9404-4e192964e49a-kube-api-access-456dc\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-m2gdn\" (UID: \"2edbf134-a1ed-436a-9404-4e192964e49a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"
May 06 17:17:50.883299 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.883276    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2edbf134-a1ed-436a-9404-4e192964e49a-extensions-socket-volume\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-m2gdn\" (UID: \"2edbf134-a1ed-436a-9404-4e192964e49a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"
May 06 17:17:50.911064 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.911035    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-456dc\" (UniqueName: \"kubernetes.io/projected/2edbf134-a1ed-436a-9404-4e192964e49a-kube-api-access-456dc\") pod \"kuadrant-operator-controller-manager-5f895dd7d5-m2gdn\" (UID: \"2edbf134-a1ed-436a-9404-4e192964e49a\") " pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"
May 06 17:17:50.979203 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:17:50.979143    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"
May 06 17:18:03.945799 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:03.945778    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"
May 06 17:18:03.947959 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:03.947932    2560 status_manager.go:895] "Failed to get status for pod" podUID="b70d58c0-5daa-4064-99fc-195823bd93b4" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mckvd\" is forbidden: User \"system:node:ip-10-0-130-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-186.ec2.internal' and this object"
May 06 17:18:03.958401 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:03.958378    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"]
May 06 17:18:03.965380 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:18:03.965349    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2edbf134_a1ed_436a_9404_4e192964e49a.slice/crio-c28b948d7c56325e94a4f68605b08f622b405313707d6c0921d1c35019d993d9 WatchSource:0}: Error finding container c28b948d7c56325e94a4f68605b08f622b405313707d6c0921d1c35019d993d9: Status 404 returned error can't find the container with id c28b948d7c56325e94a4f68605b08f622b405313707d6c0921d1c35019d993d9
May 06 17:18:04.108619 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.108532    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lx2vn\" (UniqueName: \"kubernetes.io/projected/b70d58c0-5daa-4064-99fc-195823bd93b4-kube-api-access-lx2vn\") pod \"b70d58c0-5daa-4064-99fc-195823bd93b4\" (UID: \"b70d58c0-5daa-4064-99fc-195823bd93b4\") "
May 06 17:18:04.108619 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.108579    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70d58c0-5daa-4064-99fc-195823bd93b4-extensions-socket-volume\") pod \"b70d58c0-5daa-4064-99fc-195823bd93b4\" (UID: \"b70d58c0-5daa-4064-99fc-195823bd93b4\") "
May 06 17:18:04.109088 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.109064    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/b70d58c0-5daa-4064-99fc-195823bd93b4-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "b70d58c0-5daa-4064-99fc-195823bd93b4" (UID: "b70d58c0-5daa-4064-99fc-195823bd93b4"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:18:04.110703 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.110675    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b70d58c0-5daa-4064-99fc-195823bd93b4-kube-api-access-lx2vn" (OuterVolumeSpecName: "kube-api-access-lx2vn") pod "b70d58c0-5daa-4064-99fc-195823bd93b4" (UID: "b70d58c0-5daa-4064-99fc-195823bd93b4"). InnerVolumeSpecName "kube-api-access-lx2vn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:18:04.210082 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.210050    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lx2vn\" (UniqueName: \"kubernetes.io/projected/b70d58c0-5daa-4064-99fc-195823bd93b4-kube-api-access-lx2vn\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:18:04.210082 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.210079    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/b70d58c0-5daa-4064-99fc-195823bd93b4-extensions-socket-volume\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:18:04.743198 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.743145    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn" event={"ID":"2edbf134-a1ed-436a-9404-4e192964e49a","Type":"ContainerStarted","Data":"22933dadfd42742f67a2722f24c1c6bc98bd04a248edad67ed26a34cad0ea0c2"}
May 06 17:18:04.743198 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.743201    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn" event={"ID":"2edbf134-a1ed-436a-9404-4e192964e49a","Type":"ContainerStarted","Data":"c28b948d7c56325e94a4f68605b08f622b405313707d6c0921d1c35019d993d9"}
May 06 17:18:04.743453 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.743240    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"
May 06 17:18:04.744339 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.744308    2560 generic.go:358] "Generic (PLEG): container finished" podID="b70d58c0-5daa-4064-99fc-195823bd93b4" containerID="c247e525e31245fa7c5ddf78e1b2dc4da4ccfdce6c96385890924bb7ac3c4c93" exitCode=0
May 06 17:18:04.744466 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.744344    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd"
May 06 17:18:04.744466 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.744391    2560 scope.go:117] "RemoveContainer" containerID="c247e525e31245fa7c5ddf78e1b2dc4da4ccfdce6c96385890924bb7ac3c4c93"
May 06 17:18:04.745851 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.745827    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4" event={"ID":"80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f","Type":"ContainerStarted","Data":"c85c1b9ced3cbe6560b68d38424840cb45abbd043cd70dd14cba91780a709c2c"}
May 06 17:18:04.753036 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.753020    2560 scope.go:117] "RemoveContainer" containerID="c247e525e31245fa7c5ddf78e1b2dc4da4ccfdce6c96385890924bb7ac3c4c93"
May 06 17:18:04.753305 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:18:04.753287    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c247e525e31245fa7c5ddf78e1b2dc4da4ccfdce6c96385890924bb7ac3c4c93\": container with ID starting with c247e525e31245fa7c5ddf78e1b2dc4da4ccfdce6c96385890924bb7ac3c4c93 not found: ID does not exist" containerID="c247e525e31245fa7c5ddf78e1b2dc4da4ccfdce6c96385890924bb7ac3c4c93"
May 06 17:18:04.753369 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.753312    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c247e525e31245fa7c5ddf78e1b2dc4da4ccfdce6c96385890924bb7ac3c4c93"} err="failed to get container status \"c247e525e31245fa7c5ddf78e1b2dc4da4ccfdce6c96385890924bb7ac3c4c93\": rpc error: code = NotFound desc = could not find container \"c247e525e31245fa7c5ddf78e1b2dc4da4ccfdce6c96385890924bb7ac3c4c93\": container with ID starting with c247e525e31245fa7c5ddf78e1b2dc4da4ccfdce6c96385890924bb7ac3c4c93 not found: ID does not exist"
May 06 17:18:04.767614 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.767570    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn" podStartSLOduration=14.767557176 podStartE2EDuration="14.767557176s" podCreationTimestamp="2026-05-06 17:17:50 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-06 17:18:04.765237991 +0000 UTC m=+456.209060439" watchObservedRunningTime="2026-05-06 17:18:04.767557176 +0000 UTC m=+456.211379622"
May 06 17:18:04.767948 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.767917    2560 status_manager.go:895] "Failed to get status for pod" podUID="b70d58c0-5daa-4064-99fc-195823bd93b4" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mckvd\" is forbidden: User \"system:node:ip-10-0-130-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-186.ec2.internal' and this object"
May 06 17:18:04.769838 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.769810    2560 status_manager.go:895] "Failed to get status for pod" podUID="b70d58c0-5daa-4064-99fc-195823bd93b4" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-mckvd" err="pods \"kuadrant-operator-controller-manager-5f895dd7d5-mckvd\" is forbidden: User \"system:node:ip-10-0-130-186.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"kuadrant-system\": no relationship found between node 'ip-10-0-130-186.ec2.internal' and this object"
May 06 17:18:04.787482 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:04.787440    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/kuadrant-console-plugin-6cb54b5c86-gtpm4" podStartSLOduration=2.09069104 podStartE2EDuration="25.787427881s" podCreationTimestamp="2026-05-06 17:17:39 +0000 UTC" firstStartedPulling="2026-05-06 17:17:40.224616207 +0000 UTC m=+431.668438644" lastFinishedPulling="2026-05-06 17:18:03.921353045 +0000 UTC m=+455.365175485" observedRunningTime="2026-05-06 17:18:04.785416293 +0000 UTC m=+456.229238754" watchObservedRunningTime="2026-05-06 17:18:04.787427881 +0000 UTC m=+456.231250365"
May 06 17:18:05.122483 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:05.122390    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b70d58c0-5daa-4064-99fc-195823bd93b4" path="/var/lib/kubelet/pods/b70d58c0-5daa-4064-99fc-195823bd93b4/volumes"
May 06 17:18:07.439125 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:07.439091    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"]
May 06 17:18:07.439494 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:07.439334    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn" podUID="2edbf134-a1ed-436a-9404-4e192964e49a" containerName="manager" containerID="cri-o://22933dadfd42742f67a2722f24c1c6bc98bd04a248edad67ed26a34cad0ea0c2" gracePeriod=10
May 06 17:18:07.761741 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:07.761658    2560 generic.go:358] "Generic (PLEG): container finished" podID="2edbf134-a1ed-436a-9404-4e192964e49a" containerID="22933dadfd42742f67a2722f24c1c6bc98bd04a248edad67ed26a34cad0ea0c2" exitCode=0
May 06 17:18:07.761741 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:07.761728    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn" event={"ID":"2edbf134-a1ed-436a-9404-4e192964e49a","Type":"ContainerDied","Data":"22933dadfd42742f67a2722f24c1c6bc98bd04a248edad67ed26a34cad0ea0c2"}
May 06 17:18:08.204603 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:08.204581    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"
May 06 17:18:08.350648 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:08.350560    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2edbf134-a1ed-436a-9404-4e192964e49a-extensions-socket-volume\") pod \"2edbf134-a1ed-436a-9404-4e192964e49a\" (UID: \"2edbf134-a1ed-436a-9404-4e192964e49a\") "
May 06 17:18:08.350648 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:08.350599    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-456dc\" (UniqueName: \"kubernetes.io/projected/2edbf134-a1ed-436a-9404-4e192964e49a-kube-api-access-456dc\") pod \"2edbf134-a1ed-436a-9404-4e192964e49a\" (UID: \"2edbf134-a1ed-436a-9404-4e192964e49a\") "
May 06 17:18:08.351032 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:08.351004    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2edbf134-a1ed-436a-9404-4e192964e49a-extensions-socket-volume" (OuterVolumeSpecName: "extensions-socket-volume") pod "2edbf134-a1ed-436a-9404-4e192964e49a" (UID: "2edbf134-a1ed-436a-9404-4e192964e49a"). InnerVolumeSpecName "extensions-socket-volume". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 06 17:18:08.352791 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:08.352759    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2edbf134-a1ed-436a-9404-4e192964e49a-kube-api-access-456dc" (OuterVolumeSpecName: "kube-api-access-456dc") pod "2edbf134-a1ed-436a-9404-4e192964e49a" (UID: "2edbf134-a1ed-436a-9404-4e192964e49a"). InnerVolumeSpecName "kube-api-access-456dc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:18:08.452125 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:08.452093    2560 reconciler_common.go:299] "Volume detached for volume \"extensions-socket-volume\" (UniqueName: \"kubernetes.io/empty-dir/2edbf134-a1ed-436a-9404-4e192964e49a-extensions-socket-volume\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:18:08.452125 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:08.452118    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-456dc\" (UniqueName: \"kubernetes.io/projected/2edbf134-a1ed-436a-9404-4e192964e49a-kube-api-access-456dc\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:18:08.767203 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:08.767173    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"
May 06 17:18:08.767391 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:08.767170    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn" event={"ID":"2edbf134-a1ed-436a-9404-4e192964e49a","Type":"ContainerDied","Data":"c28b948d7c56325e94a4f68605b08f622b405313707d6c0921d1c35019d993d9"}
May 06 17:18:08.767391 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:08.767298    2560 scope.go:117] "RemoveContainer" containerID="22933dadfd42742f67a2722f24c1c6bc98bd04a248edad67ed26a34cad0ea0c2"
May 06 17:18:08.809514 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:08.809486    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"]
May 06 17:18:08.813021 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:08.812995    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/kuadrant-operator-controller-manager-5f895dd7d5-m2gdn"]
May 06 17:18:09.122740 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:09.122660    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2edbf134-a1ed-436a-9404-4e192964e49a" path="/var/lib/kubelet/pods/2edbf134-a1ed-436a-9404-4e192964e49a/volumes"
May 06 17:18:23.826273 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.826242    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"]
May 06 17:18:23.827059 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.826559    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2edbf134-a1ed-436a-9404-4e192964e49a" containerName="manager"
May 06 17:18:23.827059 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.826572    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="2edbf134-a1ed-436a-9404-4e192964e49a" containerName="manager"
May 06 17:18:23.827059 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.826661    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="2edbf134-a1ed-436a-9404-4e192964e49a" containerName="manager"
May 06 17:18:23.829403 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.829374    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.831655 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.831633    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"maas-default-gateway-openshift-default-dockercfg-mlbg8\""
May 06 17:18:23.843347 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.843314    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"]
May 06 17:18:23.865575 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.865555    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-workload-certs\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.865681 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.865588    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-workload-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.865681 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.865616    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f23033c8-3765-45c5-af12-2a514f7179e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.865681 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.865638    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f23033c8-3765-45c5-af12-2a514f7179e8-istio-token\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.865681 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.865657    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-istio-data\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.865826 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.865681    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sjmfz\" (UniqueName: \"kubernetes.io/projected/f23033c8-3765-45c5-af12-2a514f7179e8-kube-api-access-sjmfz\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.865826 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.865705    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-credential-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.865826 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.865761    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.865826 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.865817    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f23033c8-3765-45c5-af12-2a514f7179e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.966938 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.966905    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-workload-certs\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967133 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.966947    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-workload-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967133 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.966974    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f23033c8-3765-45c5-af12-2a514f7179e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967286 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.967191    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f23033c8-3765-45c5-af12-2a514f7179e8-istio-token\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967286 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.967249    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-istio-data\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967396 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.967304    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sjmfz\" (UniqueName: \"kubernetes.io/projected/f23033c8-3765-45c5-af12-2a514f7179e8-kube-api-access-sjmfz\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967396 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.967331    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-certs\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-workload-certs\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967396 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.967345    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-credential-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967543 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.967405    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967543 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.967414    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"workload-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-workload-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967543 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.967473    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f23033c8-3765-45c5-af12-2a514f7179e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967696 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.967589    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-data\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-istio-data\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967696 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.967665    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"credential-socket\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-credential-socket\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.967696 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.967677    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istiod-ca-cert\" (UniqueName: \"kubernetes.io/configmap/f23033c8-3765-45c5-af12-2a514f7179e8-istiod-ca-cert\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.969600 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.969567    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-envoy\" (UniqueName: \"kubernetes.io/empty-dir/f23033c8-3765-45c5-af12-2a514f7179e8-istio-envoy\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.969883 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.969865    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-podinfo\" (UniqueName: \"kubernetes.io/downward-api/f23033c8-3765-45c5-af12-2a514f7179e8-istio-podinfo\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.980323 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.980293    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"istio-token\" (UniqueName: \"kubernetes.io/projected/f23033c8-3765-45c5-af12-2a514f7179e8-istio-token\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:23.980529 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:23.980508    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sjmfz\" (UniqueName: \"kubernetes.io/projected/f23033c8-3765-45c5-af12-2a514f7179e8-kube-api-access-sjmfz\") pod \"maas-default-gateway-openshift-default-7df95f575-cjbmt\" (UID: \"f23033c8-3765-45c5-af12-2a514f7179e8\") " pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:24.140459 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:24.140364    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:24.308298 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:24.308265    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"]
May 06 17:18:24.329810 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:18:24.329780    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf23033c8_3765_45c5_af12_2a514f7179e8.slice/crio-838db740927155221769cd901e53d1901396f5b7a61c123626b0b89a1963ea05 WatchSource:0}: Error finding container 838db740927155221769cd901e53d1901396f5b7a61c123626b0b89a1963ea05: Status 404 returned error can't find the container with id 838db740927155221769cd901e53d1901396f5b7a61c123626b0b89a1963ea05
May 06 17:18:24.331892 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:24.331856    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
May 06 17:18:24.332003 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:24.331945    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
May 06 17:18:24.332003 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:24.331989    2560 kubelet_resources.go:45] "Allocatable" allocatable={"cpu":"7500m","ephemeral-storage":"114345831029","hugepages-1Gi":"0","hugepages-2Mi":"0","memory":"31236232Ki","pods":"250"}
May 06 17:18:24.826411 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:24.826372    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt" event={"ID":"f23033c8-3765-45c5-af12-2a514f7179e8","Type":"ContainerStarted","Data":"b3c4f022a1e6ecf907ddc35f7ac21eedc2a897b351efc3b0e623b07399d5bf3f"}
May 06 17:18:24.826411 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:24.826413    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt" event={"ID":"f23033c8-3765-45c5-af12-2a514f7179e8","Type":"ContainerStarted","Data":"838db740927155221769cd901e53d1901396f5b7a61c123626b0b89a1963ea05"}
May 06 17:18:24.851670 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:24.851613    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt" podStartSLOduration=1.851592661 podStartE2EDuration="1.851592661s" podCreationTimestamp="2026-05-06 17:18:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-06 17:18:24.849694636 +0000 UTC m=+476.293517094" watchObservedRunningTime="2026-05-06 17:18:24.851592661 +0000 UTC m=+476.295415109"
May 06 17:18:25.141024 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:25.140939    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:25.145505 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:25.145481    2560 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:25.830176 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:25.830129    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:25.831145 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:25.831126    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/maas-default-gateway-openshift-default-7df95f575-cjbmt"
May 06 17:18:28.723516 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:28.723484    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-kh7p5"]
May 06 17:18:28.726707 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:28.726691    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5"
May 06 17:18:28.730483 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:28.730295    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kuadrant-system\"/\"limitador-limits-config-limitador\""
May 06 17:18:28.743946 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:28.743919    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-kh7p5"]
May 06 17:18:28.807363 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:28.807300    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qjdth\" (UniqueName: \"kubernetes.io/projected/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594-kube-api-access-qjdth\") pod \"limitador-limitador-7d549b5b-kh7p5\" (UID: \"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594\") " pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5"
May 06 17:18:28.807730 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:28.807704    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594-config-file\") pod \"limitador-limitador-7d549b5b-kh7p5\" (UID: \"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594\") " pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5"
May 06 17:18:28.810182 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:28.810142    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-kh7p5"]
May 06 17:18:28.908365 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:28.908313    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594-config-file\") pod \"limitador-limitador-7d549b5b-kh7p5\" (UID: \"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594\") " pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5"
May 06 17:18:28.908365 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:28.908369    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qjdth\" (UniqueName: \"kubernetes.io/projected/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594-kube-api-access-qjdth\") pod \"limitador-limitador-7d549b5b-kh7p5\" (UID: \"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594\") " pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5"
May 06 17:18:28.909019 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:28.908999    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594-config-file\") pod \"limitador-limitador-7d549b5b-kh7p5\" (UID: \"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594\") " pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5"
May 06 17:18:28.917955 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:28.917930    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qjdth\" (UniqueName: \"kubernetes.io/projected/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594-kube-api-access-qjdth\") pod \"limitador-limitador-7d549b5b-kh7p5\" (UID: \"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594\") " pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5"
May 06 17:18:29.041562 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:29.041487    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5"
May 06 17:18:29.209437 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:29.209401    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-kh7p5"]
May 06 17:18:29.210575 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:18:29.210547    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0c90c9ed_bfbd_4950_b7fa_75cbc0baf594.slice/crio-d9dce4dc9a215f852dc141f1cce71fe003238189f29277928cc911f1e13bfce7 WatchSource:0}: Error finding container d9dce4dc9a215f852dc141f1cce71fe003238189f29277928cc911f1e13bfce7: Status 404 returned error can't find the container with id d9dce4dc9a215f852dc141f1cce71fe003238189f29277928cc911f1e13bfce7
May 06 17:18:29.844111 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:29.844036    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5" event={"ID":"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594","Type":"ContainerStarted","Data":"d9dce4dc9a215f852dc141f1cce71fe003238189f29277928cc911f1e13bfce7"}
May 06 17:18:29.981511 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:29.981475    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kuadrant-system/authorino-7498df8756-fvx77"]
May 06 17:18:29.985440 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:29.985416    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-fvx77"
May 06 17:18:29.988381 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:29.988359    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kuadrant-system\"/\"authorino-authorino-dockercfg-k56m8\""
May 06 17:18:29.996096 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:29.996071    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-fvx77"]
May 06 17:18:30.016432 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:30.016397    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5695q\" (UniqueName: \"kubernetes.io/projected/b8d4bb55-4428-40b5-b7ff-277b898380f7-kube-api-access-5695q\") pod \"authorino-7498df8756-fvx77\" (UID: \"b8d4bb55-4428-40b5-b7ff-277b898380f7\") " pod="kuadrant-system/authorino-7498df8756-fvx77"
May 06 17:18:30.117792 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:30.117707    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5695q\" (UniqueName: \"kubernetes.io/projected/b8d4bb55-4428-40b5-b7ff-277b898380f7-kube-api-access-5695q\") pod \"authorino-7498df8756-fvx77\" (UID: \"b8d4bb55-4428-40b5-b7ff-277b898380f7\") " pod="kuadrant-system/authorino-7498df8756-fvx77"
May 06 17:18:30.129662 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:30.129630    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5695q\" (UniqueName: \"kubernetes.io/projected/b8d4bb55-4428-40b5-b7ff-277b898380f7-kube-api-access-5695q\") pod \"authorino-7498df8756-fvx77\" (UID: \"b8d4bb55-4428-40b5-b7ff-277b898380f7\") " pod="kuadrant-system/authorino-7498df8756-fvx77"
May 06 17:18:30.301012 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:30.300976    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-fvx77"
May 06 17:18:30.462470 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:30.462414    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kuadrant-system/authorino-7498df8756-fvx77"]
May 06 17:18:30.464227 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:18:30.464009    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podb8d4bb55_4428_40b5_b7ff_277b898380f7.slice/crio-5589005896f3a953015a2ce4f1306ebbe12edec04ac5fb25a7c063ac6265ef03 WatchSource:0}: Error finding container 5589005896f3a953015a2ce4f1306ebbe12edec04ac5fb25a7c063ac6265ef03: Status 404 returned error can't find the container with id 5589005896f3a953015a2ce4f1306ebbe12edec04ac5fb25a7c063ac6265ef03
May 06 17:18:30.849326 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:30.849280    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-fvx77" event={"ID":"b8d4bb55-4428-40b5-b7ff-277b898380f7","Type":"ContainerStarted","Data":"5589005896f3a953015a2ce4f1306ebbe12edec04ac5fb25a7c063ac6265ef03"}
May 06 17:18:32.860384 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:32.860341    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5" event={"ID":"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594","Type":"ContainerStarted","Data":"ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc"}
May 06 17:18:32.860827 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:32.860460    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5"
May 06 17:18:32.890985 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:32.890936    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5" podStartSLOduration=1.961706078 podStartE2EDuration="4.890918638s" podCreationTimestamp="2026-05-06 17:18:28 +0000 UTC" firstStartedPulling="2026-05-06 17:18:29.21246986 +0000 UTC m=+480.656292288" lastFinishedPulling="2026-05-06 17:18:32.141682418 +0000 UTC m=+483.585504848" observedRunningTime="2026-05-06 17:18:32.888834678 +0000 UTC m=+484.332657127" watchObservedRunningTime="2026-05-06 17:18:32.890918638 +0000 UTC m=+484.334741088"
May 06 17:18:35.873240 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:35.873201    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-fvx77" event={"ID":"b8d4bb55-4428-40b5-b7ff-277b898380f7","Type":"ContainerStarted","Data":"bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94"}
May 06 17:18:35.889254 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:35.889200    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kuadrant-system/authorino-7498df8756-fvx77" podStartSLOduration=2.571890769 podStartE2EDuration="6.889185357s" podCreationTimestamp="2026-05-06 17:18:29 +0000 UTC" firstStartedPulling="2026-05-06 17:18:30.466497124 +0000 UTC m=+481.910319562" lastFinishedPulling="2026-05-06 17:18:34.783791722 +0000 UTC m=+486.227614150" observedRunningTime="2026-05-06 17:18:35.887338798 +0000 UTC m=+487.331161246" watchObservedRunningTime="2026-05-06 17:18:35.889185357 +0000 UTC m=+487.333007831"
May 06 17:18:43.713803 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:43.713763    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-kh7p5"]
May 06 17:18:43.714287 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:43.713994    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5" podUID="0c90c9ed-bfbd-4950-b7fa-75cbc0baf594" containerName="limitador" containerID="cri-o://ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc" gracePeriod=30
May 06 17:18:43.714803 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:43.714783    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5"
May 06 17:18:44.254442 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.254418    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5"
May 06 17:18:44.340457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.340380    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594-config-file\") pod \"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594\" (UID: \"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594\") "
May 06 17:18:44.340594 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.340464    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qjdth\" (UniqueName: \"kubernetes.io/projected/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594-kube-api-access-qjdth\") pod \"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594\" (UID: \"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594\") "
May 06 17:18:44.340729 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.340707    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594-config-file" (OuterVolumeSpecName: "config-file") pod "0c90c9ed-bfbd-4950-b7fa-75cbc0baf594" (UID: "0c90c9ed-bfbd-4950-b7fa-75cbc0baf594"). InnerVolumeSpecName "config-file". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 06 17:18:44.342461 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.342429    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594-kube-api-access-qjdth" (OuterVolumeSpecName: "kube-api-access-qjdth") pod "0c90c9ed-bfbd-4950-b7fa-75cbc0baf594" (UID: "0c90c9ed-bfbd-4950-b7fa-75cbc0baf594"). InnerVolumeSpecName "kube-api-access-qjdth". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:18:44.442011 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.441981    2560 reconciler_common.go:299] "Volume detached for volume \"config-file\" (UniqueName: \"kubernetes.io/configmap/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594-config-file\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:18:44.442011 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.442007    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qjdth\" (UniqueName: \"kubernetes.io/projected/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594-kube-api-access-qjdth\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:18:44.903952 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.903916    2560 generic.go:358] "Generic (PLEG): container finished" podID="0c90c9ed-bfbd-4950-b7fa-75cbc0baf594" containerID="ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc" exitCode=0
May 06 17:18:44.904447 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.903982    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5"
May 06 17:18:44.904447 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.904005    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5" event={"ID":"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594","Type":"ContainerDied","Data":"ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc"}
May 06 17:18:44.904447 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.904056    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/limitador-limitador-7d549b5b-kh7p5" event={"ID":"0c90c9ed-bfbd-4950-b7fa-75cbc0baf594","Type":"ContainerDied","Data":"d9dce4dc9a215f852dc141f1cce71fe003238189f29277928cc911f1e13bfce7"}
May 06 17:18:44.904447 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.904078    2560 scope.go:117] "RemoveContainer" containerID="ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc"
May 06 17:18:44.912314 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.912298    2560 scope.go:117] "RemoveContainer" containerID="ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc"
May 06 17:18:44.912565 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:18:44.912547    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc\": container with ID starting with ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc not found: ID does not exist" containerID="ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc"
May 06 17:18:44.912608 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.912573    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc"} err="failed to get container status \"ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc\": rpc error: code = NotFound desc = could not find container \"ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc\": container with ID starting with ae1210465be97094386ad2b85bc6b3c055b32469d44fdb8c43b410e303dff0fc not found: ID does not exist"
May 06 17:18:44.926085 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.926058    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-kh7p5"]
May 06 17:18:44.928693 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:44.928668    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/limitador-limitador-7d549b5b-kh7p5"]
May 06 17:18:45.122162 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:45.122120    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0c90c9ed-bfbd-4950-b7fa-75cbc0baf594" path="/var/lib/kubelet/pods/0c90c9ed-bfbd-4950-b7fa-75cbc0baf594/volumes"
May 06 17:18:49.575031 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.574999    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/postgres-868db5846d-jfqqx"]
May 06 17:18:49.575517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.575451    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0c90c9ed-bfbd-4950-b7fa-75cbc0baf594" containerName="limitador"
May 06 17:18:49.575517 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.575470    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="0c90c9ed-bfbd-4950-b7fa-75cbc0baf594" containerName="limitador"
May 06 17:18:49.575600 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.575577    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="0c90c9ed-bfbd-4950-b7fa-75cbc0baf594" containerName="limitador"
May 06 17:18:49.577479 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.577454    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-868db5846d-jfqqx"
May 06 17:18:49.580288 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.580264    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"default-dockercfg-d6q8f\""
May 06 17:18:49.580413 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.580266    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"postgres-creds\""
May 06 17:18:49.589675 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.589649    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-868db5846d-jfqqx"]
May 06 17:18:49.693406 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.693371    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/18089601-8295-4947-b83e-02975d193a23-data\") pod \"postgres-868db5846d-jfqqx\" (UID: \"18089601-8295-4947-b83e-02975d193a23\") " pod="opendatahub/postgres-868db5846d-jfqqx"
May 06 17:18:49.693578 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.693471    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p28sf\" (UniqueName: \"kubernetes.io/projected/18089601-8295-4947-b83e-02975d193a23-kube-api-access-p28sf\") pod \"postgres-868db5846d-jfqqx\" (UID: \"18089601-8295-4947-b83e-02975d193a23\") " pod="opendatahub/postgres-868db5846d-jfqqx"
May 06 17:18:49.794468 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.794428    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p28sf\" (UniqueName: \"kubernetes.io/projected/18089601-8295-4947-b83e-02975d193a23-kube-api-access-p28sf\") pod \"postgres-868db5846d-jfqqx\" (UID: \"18089601-8295-4947-b83e-02975d193a23\") " pod="opendatahub/postgres-868db5846d-jfqqx"
May 06 17:18:49.794613 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.794479    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/18089601-8295-4947-b83e-02975d193a23-data\") pod \"postgres-868db5846d-jfqqx\" (UID: \"18089601-8295-4947-b83e-02975d193a23\") " pod="opendatahub/postgres-868db5846d-jfqqx"
May 06 17:18:49.794809 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.794794    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data\" (UniqueName: \"kubernetes.io/empty-dir/18089601-8295-4947-b83e-02975d193a23-data\") pod \"postgres-868db5846d-jfqqx\" (UID: \"18089601-8295-4947-b83e-02975d193a23\") " pod="opendatahub/postgres-868db5846d-jfqqx"
May 06 17:18:49.804356 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.804327    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p28sf\" (UniqueName: \"kubernetes.io/projected/18089601-8295-4947-b83e-02975d193a23-kube-api-access-p28sf\") pod \"postgres-868db5846d-jfqqx\" (UID: \"18089601-8295-4947-b83e-02975d193a23\") " pod="opendatahub/postgres-868db5846d-jfqqx"
May 06 17:18:49.888569 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:49.888489    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/postgres-868db5846d-jfqqx"
May 06 17:18:50.233256 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:50.233224    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/postgres-868db5846d-jfqqx"]
May 06 17:18:50.234011 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:18:50.233986    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18089601_8295_4947_b83e_02975d193a23.slice/crio-909afffd9da3ac4ea865df5c0cef5bba74ba3355e0c3321d362a758a26f26294 WatchSource:0}: Error finding container 909afffd9da3ac4ea865df5c0cef5bba74ba3355e0c3321d362a758a26f26294: Status 404 returned error can't find the container with id 909afffd9da3ac4ea865df5c0cef5bba74ba3355e0c3321d362a758a26f26294
May 06 17:18:50.927979 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:50.927938    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-868db5846d-jfqqx" event={"ID":"18089601-8295-4947-b83e-02975d193a23","Type":"ContainerStarted","Data":"909afffd9da3ac4ea865df5c0cef5bba74ba3355e0c3321d362a758a26f26294"}
May 06 17:18:56.954075 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:56.954030    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/postgres-868db5846d-jfqqx" event={"ID":"18089601-8295-4947-b83e-02975d193a23","Type":"ContainerStarted","Data":"83b3a7ab273af10572857dc1c082023809d662565dc22ef48b678a885e1b4e72"}
May 06 17:18:56.954479 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:56.954122    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/postgres-868db5846d-jfqqx"
May 06 17:18:56.977525 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:18:56.977484    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/postgres-868db5846d-jfqqx" podStartSLOduration=1.641431979 podStartE2EDuration="7.977471167s" podCreationTimestamp="2026-05-06 17:18:49 +0000 UTC" firstStartedPulling="2026-05-06 17:18:50.235335414 +0000 UTC m=+501.679157842" lastFinishedPulling="2026-05-06 17:18:56.571374586 +0000 UTC m=+508.015197030" observedRunningTime="2026-05-06 17:18:56.976130996 +0000 UTC m=+508.419953470" watchObservedRunningTime="2026-05-06 17:18:56.977471167 +0000 UTC m=+508.421293614"
May 06 17:19:02.985927 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:02.985898    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/postgres-868db5846d-jfqqx"
May 06 17:19:05.641803 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:05.641770    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-fvx77"]
May 06 17:19:05.642170 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:05.641995    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="kuadrant-system/authorino-7498df8756-fvx77" podUID="b8d4bb55-4428-40b5-b7ff-277b898380f7" containerName="authorino" containerID="cri-o://bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94" gracePeriod=30
May 06 17:19:05.901514 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:05.901462    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-fvx77"
May 06 17:19:05.988944 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:05.988909    2560 generic.go:358] "Generic (PLEG): container finished" podID="b8d4bb55-4428-40b5-b7ff-277b898380f7" containerID="bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94" exitCode=0
May 06 17:19:05.989124 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:05.988961    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="kuadrant-system/authorino-7498df8756-fvx77"
May 06 17:19:05.989124 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:05.989000    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-fvx77" event={"ID":"b8d4bb55-4428-40b5-b7ff-277b898380f7","Type":"ContainerDied","Data":"bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94"}
May 06 17:19:05.989124 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:05.989033    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kuadrant-system/authorino-7498df8756-fvx77" event={"ID":"b8d4bb55-4428-40b5-b7ff-277b898380f7","Type":"ContainerDied","Data":"5589005896f3a953015a2ce4f1306ebbe12edec04ac5fb25a7c063ac6265ef03"}
May 06 17:19:05.989124 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:05.989048    2560 scope.go:117] "RemoveContainer" containerID="bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94"
May 06 17:19:05.997845 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:05.997831    2560 scope.go:117] "RemoveContainer" containerID="bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94"
May 06 17:19:05.998061 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:05.998043    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94\": container with ID starting with bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94 not found: ID does not exist" containerID="bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94"
May 06 17:19:05.998113 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:05.998071    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94"} err="failed to get container status \"bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94\": rpc error: code = NotFound desc = could not find container \"bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94\": container with ID starting with bb49e4cc10f10c36475abfdafbf9fef98a68dbc4f83f075e64d0a5713ab2ea94 not found: ID does not exist"
May 06 17:19:06.042133 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.042100    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5695q\" (UniqueName: \"kubernetes.io/projected/b8d4bb55-4428-40b5-b7ff-277b898380f7-kube-api-access-5695q\") pod \"b8d4bb55-4428-40b5-b7ff-277b898380f7\" (UID: \"b8d4bb55-4428-40b5-b7ff-277b898380f7\") "
May 06 17:19:06.044144 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.044111    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b8d4bb55-4428-40b5-b7ff-277b898380f7-kube-api-access-5695q" (OuterVolumeSpecName: "kube-api-access-5695q") pod "b8d4bb55-4428-40b5-b7ff-277b898380f7" (UID: "b8d4bb55-4428-40b5-b7ff-277b898380f7"). InnerVolumeSpecName "kube-api-access-5695q". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:19:06.142761 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.142722    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5695q\" (UniqueName: \"kubernetes.io/projected/b8d4bb55-4428-40b5-b7ff-277b898380f7-kube-api-access-5695q\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:19:06.318240 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.318210    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["kuadrant-system/authorino-7498df8756-fvx77"]
May 06 17:19:06.321828 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.321804    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["kuadrant-system/authorino-7498df8756-fvx77"]
May 06 17:19:06.486001 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.485914    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-7cdc884d7c-bhpnj"]
May 06 17:19:06.486343 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.486318    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b8d4bb55-4428-40b5-b7ff-277b898380f7" containerName="authorino"
May 06 17:19:06.486343 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.486341    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="b8d4bb55-4428-40b5-b7ff-277b898380f7" containerName="authorino"
May 06 17:19:06.486531 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.486416    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="b8d4bb55-4428-40b5-b7ff-277b898380f7" containerName="authorino"
May 06 17:19:06.535855 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.535821    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cdc884d7c-bhpnj"]
May 06 17:19:06.536010 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.535958    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cdc884d7c-bhpnj"
May 06 17:19:06.539600 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.539572    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-controller-dockercfg-d8g5b\""
May 06 17:19:06.539732 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.539586    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"opendatahub\"/\"maas-parameters\""
May 06 17:19:06.646316 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.646279    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dkw2d\" (UniqueName: \"kubernetes.io/projected/3e1017da-af51-4170-a275-2a7664ef73d2-kube-api-access-dkw2d\") pod \"maas-controller-7cdc884d7c-bhpnj\" (UID: \"3e1017da-af51-4170-a275-2a7664ef73d2\") " pod="opendatahub/maas-controller-7cdc884d7c-bhpnj"
May 06 17:19:06.652004 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.651976    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-6c7675cfff-954m4"]
May 06 17:19:06.691273 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.691238    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-6c7675cfff-954m4"]
May 06 17:19:06.691423 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.691384    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c7675cfff-954m4"
May 06 17:19:06.747178 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.747073    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dkw2d\" (UniqueName: \"kubernetes.io/projected/3e1017da-af51-4170-a275-2a7664ef73d2-kube-api-access-dkw2d\") pod \"maas-controller-7cdc884d7c-bhpnj\" (UID: \"3e1017da-af51-4170-a275-2a7664ef73d2\") " pod="opendatahub/maas-controller-7cdc884d7c-bhpnj"
May 06 17:19:06.755407 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.755370    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dkw2d\" (UniqueName: \"kubernetes.io/projected/3e1017da-af51-4170-a275-2a7664ef73d2-kube-api-access-dkw2d\") pod \"maas-controller-7cdc884d7c-bhpnj\" (UID: \"3e1017da-af51-4170-a275-2a7664ef73d2\") " pod="opendatahub/maas-controller-7cdc884d7c-bhpnj"
May 06 17:19:06.785990 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.785948    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c7675cfff-954m4"]
May 06 17:19:06.786280 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:06.786257    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[kube-api-access-lf4d9], unattached volumes=[], failed to process volumes=[]: context canceled" pod="opendatahub/maas-controller-6c7675cfff-954m4" podUID="640f3953-95c7-4f6e-a5dc-e144853812d3"
May 06 17:19:06.815192 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.815162    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-586d995684-bfv5z"]
May 06 17:19:06.841687 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.841657    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-586d995684-bfv5z"]
May 06 17:19:06.841829 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.841781    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-586d995684-bfv5z"
May 06 17:19:06.845917 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.845886    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cdc884d7c-bhpnj"
May 06 17:19:06.847912 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.847885    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lf4d9\" (UniqueName: \"kubernetes.io/projected/640f3953-95c7-4f6e-a5dc-e144853812d3-kube-api-access-lf4d9\") pod \"maas-controller-6c7675cfff-954m4\" (UID: \"640f3953-95c7-4f6e-a5dc-e144853812d3\") " pod="opendatahub/maas-controller-6c7675cfff-954m4"
May 06 17:19:06.948910 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.948870    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-96wm9\" (UniqueName: \"kubernetes.io/projected/648524bc-f8b7-4d2c-917d-7d03cd3e1162-kube-api-access-96wm9\") pod \"maas-controller-586d995684-bfv5z\" (UID: \"648524bc-f8b7-4d2c-917d-7d03cd3e1162\") " pod="opendatahub/maas-controller-586d995684-bfv5z"
May 06 17:19:06.949054 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.948954    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lf4d9\" (UniqueName: \"kubernetes.io/projected/640f3953-95c7-4f6e-a5dc-e144853812d3-kube-api-access-lf4d9\") pod \"maas-controller-6c7675cfff-954m4\" (UID: \"640f3953-95c7-4f6e-a5dc-e144853812d3\") " pod="opendatahub/maas-controller-6c7675cfff-954m4"
May 06 17:19:06.958362 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.958332    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lf4d9\" (UniqueName: \"kubernetes.io/projected/640f3953-95c7-4f6e-a5dc-e144853812d3-kube-api-access-lf4d9\") pod \"maas-controller-6c7675cfff-954m4\" (UID: \"640f3953-95c7-4f6e-a5dc-e144853812d3\") " pod="opendatahub/maas-controller-6c7675cfff-954m4"
May 06 17:19:06.982292 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.982267    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-7cdc884d7c-bhpnj"]
May 06 17:19:06.982946 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:19:06.982921    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3e1017da_af51_4170_a275_2a7664ef73d2.slice/crio-4e063b1f7f5b23221baf347822f2e284ab1ed215ddcbe4f2e2877e2860be7b77 WatchSource:0}: Error finding container 4e063b1f7f5b23221baf347822f2e284ab1ed215ddcbe4f2e2877e2860be7b77: Status 404 returned error can't find the container with id 4e063b1f7f5b23221baf347822f2e284ab1ed215ddcbe4f2e2877e2860be7b77
May 06 17:19:06.993834 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.993805    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cdc884d7c-bhpnj" event={"ID":"3e1017da-af51-4170-a275-2a7664ef73d2","Type":"ContainerStarted","Data":"4e063b1f7f5b23221baf347822f2e284ab1ed215ddcbe4f2e2877e2860be7b77"}
May 06 17:19:06.994719 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.994703    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c7675cfff-954m4"
May 06 17:19:06.999294 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:06.999233    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c7675cfff-954m4"
May 06 17:19:07.049758 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:07.049721    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-96wm9\" (UniqueName: \"kubernetes.io/projected/648524bc-f8b7-4d2c-917d-7d03cd3e1162-kube-api-access-96wm9\") pod \"maas-controller-586d995684-bfv5z\" (UID: \"648524bc-f8b7-4d2c-917d-7d03cd3e1162\") " pod="opendatahub/maas-controller-586d995684-bfv5z"
May 06 17:19:07.059184 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:07.059144    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-96wm9\" (UniqueName: \"kubernetes.io/projected/648524bc-f8b7-4d2c-917d-7d03cd3e1162-kube-api-access-96wm9\") pod \"maas-controller-586d995684-bfv5z\" (UID: \"648524bc-f8b7-4d2c-917d-7d03cd3e1162\") " pod="opendatahub/maas-controller-586d995684-bfv5z"
May 06 17:19:07.121499 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:07.121471    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b8d4bb55-4428-40b5-b7ff-277b898380f7" path="/var/lib/kubelet/pods/b8d4bb55-4428-40b5-b7ff-277b898380f7/volumes"
May 06 17:19:07.150579 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:07.150554    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lf4d9\" (UniqueName: \"kubernetes.io/projected/640f3953-95c7-4f6e-a5dc-e144853812d3-kube-api-access-lf4d9\") pod \"640f3953-95c7-4f6e-a5dc-e144853812d3\" (UID: \"640f3953-95c7-4f6e-a5dc-e144853812d3\") "
May 06 17:19:07.152877 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:07.152857    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-586d995684-bfv5z"
May 06 17:19:07.152974 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:07.152854    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/640f3953-95c7-4f6e-a5dc-e144853812d3-kube-api-access-lf4d9" (OuterVolumeSpecName: "kube-api-access-lf4d9") pod "640f3953-95c7-4f6e-a5dc-e144853812d3" (UID: "640f3953-95c7-4f6e-a5dc-e144853812d3"). InnerVolumeSpecName "kube-api-access-lf4d9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:19:07.251821 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:07.251734    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lf4d9\" (UniqueName: \"kubernetes.io/projected/640f3953-95c7-4f6e-a5dc-e144853812d3-kube-api-access-lf4d9\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:19:07.307146 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:07.307121    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-586d995684-bfv5z"]
May 06 17:19:07.308747 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:19:07.308724    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod648524bc_f8b7_4d2c_917d_7d03cd3e1162.slice/crio-659ea97b866f325157152399e1d3136eb201e1dd6bdcc1c4b56c3046077dfec7 WatchSource:0}: Error finding container 659ea97b866f325157152399e1d3136eb201e1dd6bdcc1c4b56c3046077dfec7: Status 404 returned error can't find the container with id 659ea97b866f325157152399e1d3136eb201e1dd6bdcc1c4b56c3046077dfec7
May 06 17:19:08.000516 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:08.000472    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-586d995684-bfv5z" event={"ID":"648524bc-f8b7-4d2c-917d-7d03cd3e1162","Type":"ContainerStarted","Data":"659ea97b866f325157152399e1d3136eb201e1dd6bdcc1c4b56c3046077dfec7"}
May 06 17:19:08.000952 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:08.000557    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-6c7675cfff-954m4"
May 06 17:19:08.052313 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:08.051421    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-6c7675cfff-954m4"]
May 06 17:19:08.056350 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:08.056323    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-6c7675cfff-954m4"]
May 06 17:19:09.123999 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:09.123965    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="640f3953-95c7-4f6e-a5dc-e144853812d3" path="/var/lib/kubelet/pods/640f3953-95c7-4f6e-a5dc-e144853812d3/volumes"
May 06 17:19:11.015282 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:11.015240    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-586d995684-bfv5z" event={"ID":"648524bc-f8b7-4d2c-917d-7d03cd3e1162","Type":"ContainerStarted","Data":"6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3"}
May 06 17:19:11.015795 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:11.015358    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-586d995684-bfv5z"
May 06 17:19:11.016690 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:11.016663    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cdc884d7c-bhpnj" event={"ID":"3e1017da-af51-4170-a275-2a7664ef73d2","Type":"ContainerStarted","Data":"f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f"}
May 06 17:19:11.016881 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:11.016783    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-7cdc884d7c-bhpnj"
May 06 17:19:11.040728 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:11.040682    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-586d995684-bfv5z" podStartSLOduration=1.7355206600000002 podStartE2EDuration="5.040668405s" podCreationTimestamp="2026-05-06 17:19:06 +0000 UTC" firstStartedPulling="2026-05-06 17:19:07.310056588 +0000 UTC m=+518.753879013" lastFinishedPulling="2026-05-06 17:19:10.615204322 +0000 UTC m=+522.059026758" observedRunningTime="2026-05-06 17:19:11.038939059 +0000 UTC m=+522.482761507" watchObservedRunningTime="2026-05-06 17:19:11.040668405 +0000 UTC m=+522.484490851"
May 06 17:19:11.068273 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:11.068195    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-7cdc884d7c-bhpnj" podStartSLOduration=1.444740353 podStartE2EDuration="5.06818169s" podCreationTimestamp="2026-05-06 17:19:06 +0000 UTC" firstStartedPulling="2026-05-06 17:19:06.984336894 +0000 UTC m=+518.428159333" lastFinishedPulling="2026-05-06 17:19:10.607778245 +0000 UTC m=+522.051600670" observedRunningTime="2026-05-06 17:19:11.064775122 +0000 UTC m=+522.508597569" watchObservedRunningTime="2026-05-06 17:19:11.06818169 +0000 UTC m=+522.512004138"
May 06 17:19:13.061032 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:13.060986    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-7f4595bb66-9kzqf"]
May 06 17:19:13.066270 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:13.066245    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7f4595bb66-9kzqf"
May 06 17:19:13.068909 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:13.068890    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-serving-cert\""
May 06 17:19:13.090268 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:13.090243    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7f4595bb66-9kzqf"]
May 06 17:19:13.208297 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:13.208261    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/481d4898-bc40-46ab-8131-f09a9cb71be4-maas-api-tls\") pod \"maas-api-7f4595bb66-9kzqf\" (UID: \"481d4898-bc40-46ab-8131-f09a9cb71be4\") " pod="opendatahub/maas-api-7f4595bb66-9kzqf"
May 06 17:19:13.208464 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:13.208311    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-h4psz\" (UniqueName: \"kubernetes.io/projected/481d4898-bc40-46ab-8131-f09a9cb71be4-kube-api-access-h4psz\") pod \"maas-api-7f4595bb66-9kzqf\" (UID: \"481d4898-bc40-46ab-8131-f09a9cb71be4\") " pod="opendatahub/maas-api-7f4595bb66-9kzqf"
May 06 17:19:13.309291 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:13.309255    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/481d4898-bc40-46ab-8131-f09a9cb71be4-maas-api-tls\") pod \"maas-api-7f4595bb66-9kzqf\" (UID: \"481d4898-bc40-46ab-8131-f09a9cb71be4\") " pod="opendatahub/maas-api-7f4595bb66-9kzqf"
May 06 17:19:13.309484 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:13.309310    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-h4psz\" (UniqueName: \"kubernetes.io/projected/481d4898-bc40-46ab-8131-f09a9cb71be4-kube-api-access-h4psz\") pod \"maas-api-7f4595bb66-9kzqf\" (UID: \"481d4898-bc40-46ab-8131-f09a9cb71be4\") " pod="opendatahub/maas-api-7f4595bb66-9kzqf"
May 06 17:19:13.311745 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:13.311690    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/481d4898-bc40-46ab-8131-f09a9cb71be4-maas-api-tls\") pod \"maas-api-7f4595bb66-9kzqf\" (UID: \"481d4898-bc40-46ab-8131-f09a9cb71be4\") " pod="opendatahub/maas-api-7f4595bb66-9kzqf"
May 06 17:19:13.317577 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:13.317556    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-h4psz\" (UniqueName: \"kubernetes.io/projected/481d4898-bc40-46ab-8131-f09a9cb71be4-kube-api-access-h4psz\") pod \"maas-api-7f4595bb66-9kzqf\" (UID: \"481d4898-bc40-46ab-8131-f09a9cb71be4\") " pod="opendatahub/maas-api-7f4595bb66-9kzqf"
May 06 17:19:13.376542 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:13.376470    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-7f4595bb66-9kzqf"
May 06 17:19:13.509848 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:13.509824    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-7f4595bb66-9kzqf"]
May 06 17:19:13.511803 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:19:13.511762    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod481d4898_bc40_46ab_8131_f09a9cb71be4.slice/crio-17430679e3515dff932015afd97ce3901dcbb5e0569f114d413fef66564b6f36 WatchSource:0}: Error finding container 17430679e3515dff932015afd97ce3901dcbb5e0569f114d413fef66564b6f36: Status 404 returned error can't find the container with id 17430679e3515dff932015afd97ce3901dcbb5e0569f114d413fef66564b6f36
May 06 17:19:14.029743 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:14.029700    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-7f4595bb66-9kzqf" event={"ID":"481d4898-bc40-46ab-8131-f09a9cb71be4","Type":"ContainerStarted","Data":"17430679e3515dff932015afd97ce3901dcbb5e0569f114d413fef66564b6f36"}
May 06 17:19:14.783798 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:14.783711    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:19:14.785207 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:14.785174    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:19:15.422015 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:15.421963    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:19:15.423792 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:15.423766    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:19:22.026952 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.026910    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-586d995684-bfv5z"
May 06 17:19:22.027417 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.027336    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-7cdc884d7c-bhpnj"
May 06 17:19:22.093230 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.093192    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cdc884d7c-bhpnj"]
May 06 17:19:22.093412 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.093375    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-7cdc884d7c-bhpnj" podUID="3e1017da-af51-4170-a275-2a7664ef73d2" containerName="manager" containerID="cri-o://f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f" gracePeriod=10
May 06 17:19:22.298033 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.297951    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-controller-67f5887d59-4c7xx"]
May 06 17:19:22.301709 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.301687    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67f5887d59-4c7xx"
May 06 17:19:22.315384 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.315358    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67f5887d59-4c7xx"]
May 06 17:19:22.338749 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.338728    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cdc884d7c-bhpnj"
May 06 17:19:22.389132 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.389101    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dkw2d\" (UniqueName: \"kubernetes.io/projected/3e1017da-af51-4170-a275-2a7664ef73d2-kube-api-access-dkw2d\") pod \"3e1017da-af51-4170-a275-2a7664ef73d2\" (UID: \"3e1017da-af51-4170-a275-2a7664ef73d2\") "
May 06 17:19:22.389326 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.389312    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6f4qs\" (UniqueName: \"kubernetes.io/projected/b78a113f-78e1-4197-a0d9-925e8bef3bd1-kube-api-access-6f4qs\") pod \"maas-controller-67f5887d59-4c7xx\" (UID: \"b78a113f-78e1-4197-a0d9-925e8bef3bd1\") " pod="opendatahub/maas-controller-67f5887d59-4c7xx"
May 06 17:19:22.391365 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.391328    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/3e1017da-af51-4170-a275-2a7664ef73d2-kube-api-access-dkw2d" (OuterVolumeSpecName: "kube-api-access-dkw2d") pod "3e1017da-af51-4170-a275-2a7664ef73d2" (UID: "3e1017da-af51-4170-a275-2a7664ef73d2"). InnerVolumeSpecName "kube-api-access-dkw2d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:19:22.489730 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.489690    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6f4qs\" (UniqueName: \"kubernetes.io/projected/b78a113f-78e1-4197-a0d9-925e8bef3bd1-kube-api-access-6f4qs\") pod \"maas-controller-67f5887d59-4c7xx\" (UID: \"b78a113f-78e1-4197-a0d9-925e8bef3bd1\") " pod="opendatahub/maas-controller-67f5887d59-4c7xx"
May 06 17:19:22.489896 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.489776    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dkw2d\" (UniqueName: \"kubernetes.io/projected/3e1017da-af51-4170-a275-2a7664ef73d2-kube-api-access-dkw2d\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:19:22.498971 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.498937    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6f4qs\" (UniqueName: \"kubernetes.io/projected/b78a113f-78e1-4197-a0d9-925e8bef3bd1-kube-api-access-6f4qs\") pod \"maas-controller-67f5887d59-4c7xx\" (UID: \"b78a113f-78e1-4197-a0d9-925e8bef3bd1\") " pod="opendatahub/maas-controller-67f5887d59-4c7xx"
May 06 17:19:22.614290 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.614203    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-67f5887d59-4c7xx"
May 06 17:19:22.745648 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:22.745616    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-controller-67f5887d59-4c7xx"]
May 06 17:19:22.746869 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:19:22.746832    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb78a113f_78e1_4197_a0d9_925e8bef3bd1.slice/crio-fc2aa2374ee5da28f338e5912ba38b5ca86e798379004364e313e17646a28501 WatchSource:0}: Error finding container fc2aa2374ee5da28f338e5912ba38b5ca86e798379004364e313e17646a28501: Status 404 returned error can't find the container with id fc2aa2374ee5da28f338e5912ba38b5ca86e798379004364e313e17646a28501
May 06 17:19:23.063577 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:23.063530    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67f5887d59-4c7xx" event={"ID":"b78a113f-78e1-4197-a0d9-925e8bef3bd1","Type":"ContainerStarted","Data":"fc2aa2374ee5da28f338e5912ba38b5ca86e798379004364e313e17646a28501"}
May 06 17:19:23.064687 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:23.064662    2560 generic.go:358] "Generic (PLEG): container finished" podID="3e1017da-af51-4170-a275-2a7664ef73d2" containerID="f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f" exitCode=0
May 06 17:19:23.064775 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:23.064700    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cdc884d7c-bhpnj" event={"ID":"3e1017da-af51-4170-a275-2a7664ef73d2","Type":"ContainerDied","Data":"f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f"}
May 06 17:19:23.064775 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:23.064729    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-7cdc884d7c-bhpnj" event={"ID":"3e1017da-af51-4170-a275-2a7664ef73d2","Type":"ContainerDied","Data":"4e063b1f7f5b23221baf347822f2e284ab1ed215ddcbe4f2e2877e2860be7b77"}
May 06 17:19:23.064775 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:23.064746    2560 scope.go:117] "RemoveContainer" containerID="f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f"
May 06 17:19:23.064775 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:23.064752    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-7cdc884d7c-bhpnj"
May 06 17:19:23.073857 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:23.073841    2560 scope.go:117] "RemoveContainer" containerID="f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f"
May 06 17:19:23.074124 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:23.074102    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f\": container with ID starting with f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f not found: ID does not exist" containerID="f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f"
May 06 17:19:23.074245 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:23.074129    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f"} err="failed to get container status \"f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f\": rpc error: code = NotFound desc = could not find container \"f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f\": container with ID starting with f11deba526256dc80461c4375e0f046e7e19041bc25ba0adae6dcc16e484fa5f not found: ID does not exist"
May 06 17:19:23.088426 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:23.088401    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-7cdc884d7c-bhpnj"]
May 06 17:19:23.093012 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:23.092989    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-7cdc884d7c-bhpnj"]
May 06 17:19:23.122418 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:23.122391    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="3e1017da-af51-4170-a275-2a7664ef73d2" path="/var/lib/kubelet/pods/3e1017da-af51-4170-a275-2a7664ef73d2/volumes"
May 06 17:19:24.070488 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:24.070457    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-67f5887d59-4c7xx" event={"ID":"b78a113f-78e1-4197-a0d9-925e8bef3bd1","Type":"ContainerStarted","Data":"189f6192a41aa64114b928b2933af4d883d4f43c6d2c96d38c9ec2ae3c20bce2"}
May 06 17:19:24.070857 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:24.070607    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="opendatahub/maas-controller-67f5887d59-4c7xx"
May 06 17:19:24.090729 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:24.090679    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="opendatahub/maas-controller-67f5887d59-4c7xx" podStartSLOduration=1.77002481 podStartE2EDuration="2.090664828s" podCreationTimestamp="2026-05-06 17:19:22 +0000 UTC" firstStartedPulling="2026-05-06 17:19:22.74809139 +0000 UTC m=+534.191913815" lastFinishedPulling="2026-05-06 17:19:23.068731404 +0000 UTC m=+534.512553833" observedRunningTime="2026-05-06 17:19:24.08809076 +0000 UTC m=+535.531913207" watchObservedRunningTime="2026-05-06 17:19:24.090664828 +0000 UTC m=+535.534487325"
May 06 17:19:28.443794 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:28.443729    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:19:28.444909 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:28.444885    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:19:35.080038 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:35.079998    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="opendatahub/maas-controller-67f5887d59-4c7xx"
May 06 17:19:35.141902 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:35.141866    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-586d995684-bfv5z"]
May 06 17:19:35.142180 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:35.142122    2560 kuberuntime_container.go:864] "Killing container with a grace period" pod="opendatahub/maas-controller-586d995684-bfv5z" podUID="648524bc-f8b7-4d2c-917d-7d03cd3e1162" containerName="manager" containerID="cri-o://6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3" gracePeriod=10
May 06 17:19:35.388427 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:35.388401    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-586d995684-bfv5z"
May 06 17:19:35.510855 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:35.510827    2560 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-96wm9\" (UniqueName: \"kubernetes.io/projected/648524bc-f8b7-4d2c-917d-7d03cd3e1162-kube-api-access-96wm9\") pod \"648524bc-f8b7-4d2c-917d-7d03cd3e1162\" (UID: \"648524bc-f8b7-4d2c-917d-7d03cd3e1162\") "
May 06 17:19:35.512754 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:35.512727    2560 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/648524bc-f8b7-4d2c-917d-7d03cd3e1162-kube-api-access-96wm9" (OuterVolumeSpecName: "kube-api-access-96wm9") pod "648524bc-f8b7-4d2c-917d-7d03cd3e1162" (UID: "648524bc-f8b7-4d2c-917d-7d03cd3e1162"). InnerVolumeSpecName "kube-api-access-96wm9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 06 17:19:35.612378 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:35.612338    2560 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-96wm9\" (UniqueName: \"kubernetes.io/projected/648524bc-f8b7-4d2c-917d-7d03cd3e1162-kube-api-access-96wm9\") on node \"ip-10-0-130-186.ec2.internal\" DevicePath \"\""
May 06 17:19:36.113035 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:36.112999    2560 generic.go:358] "Generic (PLEG): container finished" podID="648524bc-f8b7-4d2c-917d-7d03cd3e1162" containerID="6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3" exitCode=0
May 06 17:19:36.113525 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:36.113045    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-586d995684-bfv5z" event={"ID":"648524bc-f8b7-4d2c-917d-7d03cd3e1162","Type":"ContainerDied","Data":"6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3"}
May 06 17:19:36.113525 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:36.113065    2560 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-controller-586d995684-bfv5z"
May 06 17:19:36.113525 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:36.113089    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-controller-586d995684-bfv5z" event={"ID":"648524bc-f8b7-4d2c-917d-7d03cd3e1162","Type":"ContainerDied","Data":"659ea97b866f325157152399e1d3136eb201e1dd6bdcc1c4b56c3046077dfec7"}
May 06 17:19:36.113525 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:36.113105    2560 scope.go:117] "RemoveContainer" containerID="6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3"
May 06 17:19:36.121661 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:36.121644    2560 scope.go:117] "RemoveContainer" containerID="6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3"
May 06 17:19:36.121907 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:36.121889    2560 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3\": container with ID starting with 6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3 not found: ID does not exist" containerID="6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3"
May 06 17:19:36.121958 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:36.121915    2560 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3"} err="failed to get container status \"6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3\": rpc error: code = NotFound desc = could not find container \"6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3\": container with ID starting with 6f3ccaf4b4362ccbfb96e27116bafb4b72252791350fce5d6883ef9a713f86a3 not found: ID does not exist"
May 06 17:19:36.134484 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:36.134461    2560 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["opendatahub/maas-controller-586d995684-bfv5z"]
May 06 17:19:36.137986 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:36.137968    2560 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["opendatahub/maas-controller-586d995684-bfv5z"]
May 06 17:19:37.121595 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:37.121563    2560 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="648524bc-f8b7-4d2c-917d-7d03cd3e1162" path="/var/lib/kubelet/pods/648524bc-f8b7-4d2c-917d-7d03cd3e1162/volumes"
May 06 17:19:40.424524 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:40.424472    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:19:40.425660 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:40.425634    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:19:42.536997 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.536960    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["opendatahub/maas-api-5c87f7f96f-dmssh"]
May 06 17:19:42.537369 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.537347    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="3e1017da-af51-4170-a275-2a7664ef73d2" containerName="manager"
May 06 17:19:42.537369 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.537361    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="3e1017da-af51-4170-a275-2a7664ef73d2" containerName="manager"
May 06 17:19:42.537448 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.537376    2560 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="648524bc-f8b7-4d2c-917d-7d03cd3e1162" containerName="manager"
May 06 17:19:42.537448 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.537381    2560 state_mem.go:107] "Deleted CPUSet assignment" podUID="648524bc-f8b7-4d2c-917d-7d03cd3e1162" containerName="manager"
May 06 17:19:42.537448 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.537436    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="3e1017da-af51-4170-a275-2a7664ef73d2" containerName="manager"
May 06 17:19:42.537448 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.537443    2560 memory_manager.go:356] "RemoveStaleState removing state" podUID="648524bc-f8b7-4d2c-917d-7d03cd3e1162" containerName="manager"
May 06 17:19:42.541962 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.541941    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5c87f7f96f-dmssh"
May 06 17:19:42.544266 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.544248    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"opendatahub\"/\"maas-api-dockercfg-7h8kw\""
May 06 17:19:42.550980 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.550961    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5c87f7f96f-dmssh"]
May 06 17:19:42.680128 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.680093    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzx46\" (UniqueName: \"kubernetes.io/projected/de6f9aea-3484-4180-9db2-82cbf9602ce9-kube-api-access-zzx46\") pod \"maas-api-5c87f7f96f-dmssh\" (UID: \"de6f9aea-3484-4180-9db2-82cbf9602ce9\") " pod="opendatahub/maas-api-5c87f7f96f-dmssh"
May 06 17:19:42.680310 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.680223    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6f9aea-3484-4180-9db2-82cbf9602ce9-maas-api-tls\") pod \"maas-api-5c87f7f96f-dmssh\" (UID: \"de6f9aea-3484-4180-9db2-82cbf9602ce9\") " pod="opendatahub/maas-api-5c87f7f96f-dmssh"
May 06 17:19:42.781066 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.781029    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zzx46\" (UniqueName: \"kubernetes.io/projected/de6f9aea-3484-4180-9db2-82cbf9602ce9-kube-api-access-zzx46\") pod \"maas-api-5c87f7f96f-dmssh\" (UID: \"de6f9aea-3484-4180-9db2-82cbf9602ce9\") " pod="opendatahub/maas-api-5c87f7f96f-dmssh"
May 06 17:19:42.781241 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.781136    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6f9aea-3484-4180-9db2-82cbf9602ce9-maas-api-tls\") pod \"maas-api-5c87f7f96f-dmssh\" (UID: \"de6f9aea-3484-4180-9db2-82cbf9602ce9\") " pod="opendatahub/maas-api-5c87f7f96f-dmssh"
May 06 17:19:42.783581 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.783561    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"maas-api-tls\" (UniqueName: \"kubernetes.io/secret/de6f9aea-3484-4180-9db2-82cbf9602ce9-maas-api-tls\") pod \"maas-api-5c87f7f96f-dmssh\" (UID: \"de6f9aea-3484-4180-9db2-82cbf9602ce9\") " pod="opendatahub/maas-api-5c87f7f96f-dmssh"
May 06 17:19:42.788960 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.788914    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzx46\" (UniqueName: \"kubernetes.io/projected/de6f9aea-3484-4180-9db2-82cbf9602ce9-kube-api-access-zzx46\") pod \"maas-api-5c87f7f96f-dmssh\" (UID: \"de6f9aea-3484-4180-9db2-82cbf9602ce9\") " pod="opendatahub/maas-api-5c87f7f96f-dmssh"
May 06 17:19:42.852967 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.852934    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="opendatahub/maas-api-5c87f7f96f-dmssh"
May 06 17:19:42.986226 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:42.986203    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["opendatahub/maas-api-5c87f7f96f-dmssh"]
May 06 17:19:42.987740 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:19:42.987714    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podde6f9aea_3484_4180_9db2_82cbf9602ce9.slice/crio-21b3dfe90ae2c3fa9e20a9a899cafb2f42f0fa456bf76fa748e444f015fa8e93 WatchSource:0}: Error finding container 21b3dfe90ae2c3fa9e20a9a899cafb2f42f0fa456bf76fa748e444f015fa8e93: Status 404 returned error can't find the container with id 21b3dfe90ae2c3fa9e20a9a899cafb2f42f0fa456bf76fa748e444f015fa8e93
May 06 17:19:43.142572 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:19:43.142488    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="opendatahub/maas-api-5c87f7f96f-dmssh" event={"ID":"de6f9aea-3484-4180-9db2-82cbf9602ce9","Type":"ContainerStarted","Data":"21b3dfe90ae2c3fa9e20a9a899cafb2f42f0fa456bf76fa748e444f015fa8e93"}
May 06 17:19:44.520639 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:44.520584    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:19:44.521762 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:44.521738    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:19:45.507507 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:45.507451    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:19:45.508633 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:45.508610    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:19:54.453076 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:54.453024    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:19:54.454209 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:19:54.454186    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:20:00.465239 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:00.465184    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:20:00.466386 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:00.466359    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:20:09.378095 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:09.378014    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:20:09.379229 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:09.379204    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:20:11.482695 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:11.482640    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:20:11.483831 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:11.483805    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:20:22.390783 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:22.390724    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:20:22.391894 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:22.391870    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:20:23.471068 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:23.471015    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:20:23.472222 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:23.472198    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:20:34.395060 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:34.395006    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:20:34.396220 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:34.396196    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:20:38.118117 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:20:38.118086    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 06 17:20:38.482284 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:38.482232    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:20:38.483411 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:38.483385    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:20:45.370816 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:45.370766    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:20:45.371942 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:45.371919    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:20:52.434424 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:52.434371    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:20:52.435555 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:52.435531    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:20:58.415122 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:58.415045    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:20:58.416203 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:20:58.416185    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:21:07.433712 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:07.433652    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:21:07.434843 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:07.434819    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:21:13.460527 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:13.460467    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:21:13.461651 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:13.461628    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:21:21.545101 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:21.545045    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:21:21.546230 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:21.546206    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:21:25.453240 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:25.453139    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:21:25.454367 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:25.454343    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:21:36.444175 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:36.444100    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:21:36.445301 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:36.445277    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:21:39.396623 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:39.396549    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:21:39.397756 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:39.397731    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:21:48.419202 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:48.419117    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:21:48.420316 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:48.420292    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:21:53.412243 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:53.412120    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:21:53.413366 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:21:53.413345    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:22:01.407835 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:01.407778    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:22:01.408954 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:01.408931    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:22:06.338098 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:06.338040    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:22:06.339231 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:06.339207    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:22:17.483489 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:17.483406    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:22:17.484610 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:17.484587    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:22:18.401105 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:18.401051    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:22:18.402227 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:18.402203    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:22:29.436206 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:29.436132    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:22:29.437331 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:29.437307    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:22:32.408108 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:32.408047    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:22:32.409238 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:32.409214    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:22:42.557225 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:42.557146    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:22:42.558353 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:42.558330    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:22:47.338992 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:47.338940    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:22:47.340128 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:47.340101    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:22:54.490965 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:54.490910    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:22:54.492084 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:54.492060    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:22:59.382543 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:59.382447    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:22:59.383675 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:22:59.383651    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:23:10.473247 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:10.473195    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:23:10.474379 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:10.474353    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:23:12.379163 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:12.379094    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:23:12.380265 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:12.380237    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:23:22.399883 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:22.399830    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:23:22.401006 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:22.400981    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:23:28.382298 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:28.382242    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:23:28.383423 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:28.383402    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:23:37.592710 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:37.592656    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:23:37.593830 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:37.593808    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:23:44.411541 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:44.411485    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:23:44.412664 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:44.412641    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:23:52.557900 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:52.557824    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:23:52.559085 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:52.559061    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:23:57.380894 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:57.380835    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:23:57.382019 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:23:57.381994    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:24:05.421549 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:05.421497    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:24:05.422679 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:05.422661    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:24:11.412118 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:11.412060    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:24:11.413217 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:11.413193    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:24:18.539652 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:18.539597    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:24:18.540771 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:18.540753    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:24:25.567669 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:25.567616    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:24:25.568803 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:25.568781    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:24:32.475461 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:32.475387    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:24:32.476586 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:32.476564    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:24:40.346215 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:40.346136    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:24:40.347350 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:40.347325    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:24:48.425061 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:48.425005    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:24:48.426201 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:48.426178    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:24:53.403294 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:53.403220    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:24:53.404424 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:24:53.404394    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:25:03.710241 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:03.710181    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:25:03.711366 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:03.711343    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:25:05.500543 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:05.500488    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:25:05.501671 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:05.501645    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:25:15.478642 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:15.478582    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:25:15.479761 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:15.479737    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:25:20.437089 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:20.437026    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:25:20.438237 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:20.438212    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:25:27.486563 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:27.486503    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:25:27.487691 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:27.487666    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:25:31.382590 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:31.382534    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:25:31.383719 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:31.383695    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:25:42.117853 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:25:42.117819    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 06 17:25:42.418366 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:42.418314    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:25:42.419483 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:42.419462    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:25:45.500640 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:45.500585    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:25:45.501766 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:45.501746    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:25:53.513505 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:53.513450    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:25:53.514647 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:25:53.514614    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:26:00.364095 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:00.364037    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:26:00.365211 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:00.365188    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:26:05.549291 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:05.549237    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:26:05.550411 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:05.550391    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:26:15.484703 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:15.484645    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:26:15.485832 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:15.485809    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:26:16.402980 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:16.402924    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:26:16.404106 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:16.404084    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:26:30.399129 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:30.399056    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:26:30.400254 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:30.400229    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:26:32.489522 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:32.489468    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:26:32.490648 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:32.490625    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:26:42.440558 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:42.440506    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:26:42.441683 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:42.441659    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:26:45.525521 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:45.525469    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:26:45.526644 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:45.526622    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:26:56.368635 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:56.368581    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:26:56.369759 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:56.369739    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:26:56.430261 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:56.430205    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:26:56.431393 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:26:56.431371    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:27:10.435629 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:10.435573    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:27:10.436756 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:10.436732    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:27:12.379171 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:12.379105    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:27:12.380598 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:12.380576    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:27:21.439019 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:21.438953    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:27:21.440145 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:21.440121    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:27:24.411731 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:24.411669    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:27:24.412855 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:24.412833    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:27:33.504909 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:33.504858    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:27:33.506036 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:33.506015    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:27:37.397581 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:37.397530    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:27:37.398693 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:37.398674    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:27:48.438932 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:48.438873    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:27:48.440057 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:48.440033    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:27:51.388865 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:51.388809    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:27:51.390001 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:27:51.389977    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:28:00.516037 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:00.515958    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:28:00.517182 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:00.517140    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:28:06.365073 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:06.365015    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:28:06.366218 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:06.366194    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:28:12.409411 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:12.409353    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:28:12.410535 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:12.410513    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:28:19.394580 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:19.394525    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:28:19.395702 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:19.395678    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:28:23.460460 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:23.460402    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:28:23.461598 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:23.461574    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:28:34.395713 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:34.395652    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:28:34.396840 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:34.396814    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:28:35.579569 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:35.579514    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:28:35.580693 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:35.580670    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:28:47.428203 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:47.428118    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:28:47.429332 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:47.429308    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:28:48.430717 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:48.430660    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:28:48.431841 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:28:48.431817    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:29:01.380590 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:01.380535    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:29:01.381722 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:01.381696    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:29:03.520308 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:03.520255    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:29:03.521427 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:03.521404    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:29:12.525375 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:12.525318    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:29:12.526439 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:12.526409    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:29:16.418401 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:16.418344    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:29:16.419525 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:16.419500    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:29:23.367674 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:23.367599    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:29:23.368824 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:23.368776    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:29:30.499531 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:30.499477    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:29:30.500665 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:30.500641    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:29:38.416857 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:38.416798    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:29:38.417980 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:38.417955    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:29:44.454795 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:44.454731    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:29:44.455849 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:44.455825    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:29:52.397789 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:52.397715    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:29:52.399029 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:52.399006    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:29:55.452661 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:55.452593    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:29:55.453771 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:29:55.453749    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:30:06.381084 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:06.381010    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:30:06.382226 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:06.382191    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:30:09.517353 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:09.517297    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:30:09.518474 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:09.518455    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:30:19.447229 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:19.447118    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:30:19.448353 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:19.448330    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:30:23.670838 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:23.670780    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:30:23.671961 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:23.671938    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:30:31.540541 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:31.540489    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:30:31.541676 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:31.541650    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:30:35.409930 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:35.409872    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:30:35.411054 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:35.411032    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:30:46.118369 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:30:46.118337    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 06 17:30:46.517933 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:46.517880    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:30:46.519056 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:46.519033    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:30:48.572582 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:48.572511    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:30:48.573720 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:30:48.573695    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:31:01.366112 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:01.366039    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:31:01.367213 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:01.367190    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:31:02.534182 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:02.534083    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:31:02.535320 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:02.535296    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:31:15.383210 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:15.383127    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:31:15.384333 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:15.384309    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:31:17.453692 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:17.453638    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:31:17.454821 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:17.454796    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:31:30.470723 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:30.470665    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:31:30.471845 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:30.471821    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:31:31.462768 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:31.462712    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:31:31.463891 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:31.463869    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:31:42.404982 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:42.404929    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:31:42.406107 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:42.406084    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:31:46.426633 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:46.426580    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:31:46.427757 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:46.427728    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:31:57.444601 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:57.444548    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:31:57.445728 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:57.445706    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:31:59.732675 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:59.732618    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:31:59.733818 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:31:59.733792    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:32:11.488914 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:11.488861    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:32:11.490042 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:11.490017    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:32:13.427086 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:13.427032    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:32:13.428213 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:13.428190    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:32:24.391649 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:24.391596    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:32:24.392754 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:24.392735    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:32:25.588690 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:25.588633    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:32:25.590505 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:25.590476    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:32:36.589836 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:36.589778    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:32:36.590966 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:36.590941    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:32:39.422246 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:39.422194    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:32:39.424047 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:39.424022    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:32:48.466338 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:48.466281    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:32:48.467461 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:48.467441    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:32:53.448506 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:53.448450    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:32:53.449633 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:32:53.449609    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:33:01.524586 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:01.524532    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:33:01.525704 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:01.525685    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:33:04.373712 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:04.373643    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:33:04.374830 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:04.374805    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:33:12.872452 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:12.872388    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:33:12.873573 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:12.873549    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:33:19.364863 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:19.364810    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:33:19.366000 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:19.365975    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:33:24.395264 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:24.395207    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:33:24.396395 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:24.396368    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:33:34.469548 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:34.469458    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:33:34.470671 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:34.470647    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:33:36.451296 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:36.451241    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:33:36.452396 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:36.452378    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:33:47.466210 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:47.466124    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:33:47.467349 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:47.467315    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:33:51.593198 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:51.593124    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:33:51.594335 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:33:51.594299    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:34:01.531695 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:01.531642    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:34:01.532815 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:01.532793    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:34:06.515108 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:06.515054    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:34:06.516210 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:06.516188    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:34:14.435568 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:14.435514    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:34:14.436691 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:14.436665    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:34:21.500592 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:21.500534    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:34:21.501713 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:21.501693    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:34:28.435899 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:28.435844    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:34:28.437095 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:28.437071    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:34:35.497834 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:35.497766    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:34:35.498958 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:35.498932    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:34:43.461254 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:43.456701    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:34:43.461254 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:43.460597    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:34:48.452854 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:48.452796    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:34:48.453979 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:48.453960    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:34:58.399569 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:58.399504    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:34:58.400693 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:34:58.400669    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:35:00.574214 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:00.574138    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:35:00.575344 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:00.575314    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:35:10.385912 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:10.385856    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:35:10.387045 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:10.387021    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:35:12.453739 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:12.453686    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:35:12.454863 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:12.454842    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:35:25.380874 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:25.380819    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:35:25.382001 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:25.381978    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:35:26.551414 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:26.551360    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:35:26.552529 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:26.552511    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:35:37.453441 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:37.453387    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:35:37.455220 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:37.455199    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:35:39.433387 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:39.433335    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:35:39.434512 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:39.434488    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:35:49.120519 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:35:49.120488    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 06 17:35:49.555121 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:49.555068    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:35:49.556239 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:49.556215    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:35:52.477699 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:52.477617    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:35:52.478818 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:35:52.478794    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:36:02.432878 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:02.432827    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:36:02.434004 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:02.433981    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:36:04.387644 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:04.387567    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:36:04.388768 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:04.388745    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:36:17.390024 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:17.389952    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:36:17.391139 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:17.391117    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:36:17.484075 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:17.484023    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:36:17.485212 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:17.485189    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:36:29.532995 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:29.532942    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:36:29.534118 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:29.534096    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:36:33.476359 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:33.476306    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:36:33.477486 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:33.477461    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:36:42.560559 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:42.560506    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:36:42.561697 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:42.561670    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:36:47.438026 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:47.437960    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:36:47.439161 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:47.439132    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:36:56.421628 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:56.421565    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:36:56.423520 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:36:56.423502    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:37:01.420927 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:01.420855    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:37:01.422063 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:01.422037    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:37:10.382830 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:10.382765    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:37:10.383963 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:10.383939    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:37:16.458607 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:16.458530    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:37:16.459734 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:16.459707    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:37:23.382281 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:23.382224    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:37:23.383409 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:23.383386    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:37:29.449101 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:29.449046    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:37:29.450229 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:29.450207    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:37:34.427923 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:34.427872    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:37:34.429040 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:34.429020    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:37:44.442019 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:44.441963    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:37:44.443140 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:44.443118    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:37:48.342068 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:48.342014    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:37:48.343193 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:48.343171    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:37:56.429797 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:56.429742    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:37:56.430929 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:37:56.430906    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:38:00.471964 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:00.471913    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:38:00.473080 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:00.473060    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:38:11.380325 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:11.380264    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:38:11.381449 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:11.381425    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:38:12.478828 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:12.478777    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:38:12.479954 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:12.479929    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:38:23.446304 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:23.446238    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:38:23.447434 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:23.447409    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:38:26.362128 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:26.362043    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:38:26.367314 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:26.367274    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:38:36.531371 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:36.531307    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:38:36.532497 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:36.532472    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:38:38.350195 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:38.350119    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:38:38.351326 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:38.351296    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:38:51.497674 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:51.497601    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:38:51.498802 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:51.498779    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:38:53.440103 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:53.440044    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:38:53.441224 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:38:53.441198    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:39:03.493606 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:03.493548    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:39:03.494725 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:03.494705    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:39:04.378534 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:04.378477    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:39:04.379652 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:04.379629    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:39:15.432595 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:15.432538    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:39:15.433702 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:15.433682    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:39:18.569355 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:18.569299    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:39:18.570484 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:18.570452    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:39:30.379333 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:30.379259    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:39:30.380457 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:30.380435    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:39:31.473694 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:31.473634    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:39:31.474857 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:31.474798    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:39:42.387814 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:42.387748    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:39:42.388934 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:42.388909    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:39:46.572690 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:46.572637    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:39:46.573816 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:46.573793    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:39:54.480456 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:54.480400    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:39:54.481573 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:39:54.481549    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:40:01.668273 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:01.668221    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:40:01.669397 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:01.669370    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:40:03.770524 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.770488    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"]
May 06 17:40:03.774246 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.774226    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.779471 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.779449    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"default-dockercfg-mb796\""
May 06 17:40:03.780856 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.780829    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"openshift-service-ca.crt\""
May 06 17:40:03.780984 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.780833    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"e2e-distinct-2-simulated-kserve-self-signed-certs\""
May 06 17:40:03.784095 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.784079    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"llm\"/\"kube-root-ca.crt\""
May 06 17:40:03.808698 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.808675    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"]
May 06 17:40:03.859011 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.858975    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.859186 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.859019    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.859186 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.859046    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.859186 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.859092    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.859306 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.859184    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dzq8\" (UniqueName: \"kubernetes.io/projected/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-kube-api-access-5dzq8\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.859306 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.859223    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.959628 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.959588    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.959799 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.959640    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.959799 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.959673    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.959799 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.959691    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.959799 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.959706    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.959799 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.959738    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5dzq8\" (UniqueName: \"kubernetes.io/projected/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-kube-api-access-5dzq8\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.960139 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.960113    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-kserve-provision-location\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.960253 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.960135    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-model-cache\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.960253 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.960208    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-home\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.961979 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.961954    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-dshm\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.962076 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.962055    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-tls-certs\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:03.968792 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:03.968773    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dzq8\" (UniqueName: \"kubernetes.io/projected/fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c-kube-api-access-5dzq8\") pod \"e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm\" (UID: \"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c\") " pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:04.085056 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:04.084955    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:04.216327 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:04.216298    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"]
May 06 17:40:04.217829 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:40:04.217802    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfcffb244_bd53_4fb0_b73b_b4bb6b2dc37c.slice/crio-e48cee7b45317d7cc3a25f8c21547dfda37ec7957c81ca5e4050ca0f681a9b90 WatchSource:0}: Error finding container e48cee7b45317d7cc3a25f8c21547dfda37ec7957c81ca5e4050ca0f681a9b90: Status 404 returned error can't find the container with id e48cee7b45317d7cc3a25f8c21547dfda37ec7957c81ca5e4050ca0f681a9b90
May 06 17:40:04.625116 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:04.625080    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm" event={"ID":"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c","Type":"ContainerStarted","Data":"e48cee7b45317d7cc3a25f8c21547dfda37ec7957c81ca5e4050ca0f681a9b90"}
May 06 17:40:05.350651 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:05.350597    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:40:05.351774 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:05.351751    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:40:11.655705 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:11.655669    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm" event={"ID":"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c","Type":"ContainerStarted","Data":"09263706f11d971ab3370c00717c8abe2c2fbc30bcc7142da469fc72ecb24c67"}
May 06 17:40:14.691840 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:14.691781    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:40:14.692965 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:14.692940    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:40:16.677659 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:16.677628    2560 generic.go:358] "Generic (PLEG): container finished" podID="fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c" containerID="09263706f11d971ab3370c00717c8abe2c2fbc30bcc7142da469fc72ecb24c67" exitCode=0
May 06 17:40:16.678037 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:16.677713    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm" event={"ID":"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c","Type":"ContainerDied","Data":"09263706f11d971ab3370c00717c8abe2c2fbc30bcc7142da469fc72ecb24c67"}
May 06 17:40:18.366246 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:18.366193    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:40:18.367372 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:18.367349    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:40:18.687873 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:18.687839    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm" event={"ID":"fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c","Type":"ContainerStarted","Data":"f0d5545e0c35e0dba23affe7f62968857307521f27d52fed1d8c1c29ba2f956d"}
May 06 17:40:18.688088 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:18.688066    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:18.709718 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:18.709666    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm" podStartSLOduration=2.040759743 podStartE2EDuration="15.709651685s" podCreationTimestamp="2026-05-06 17:40:03 +0000 UTC" firstStartedPulling="2026-05-06 17:40:04.219522774 +0000 UTC m=+1775.663345202" lastFinishedPulling="2026-05-06 17:40:17.888414714 +0000 UTC m=+1789.332237144" observedRunningTime="2026-05-06 17:40:18.707902825 +0000 UTC m=+1790.151725273" watchObservedRunningTime="2026-05-06 17:40:18.709651685 +0000 UTC m=+1790.153474131"
May 06 17:40:24.191225 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.191187    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"]
May 06 17:40:24.194877 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.194847    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.197789 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.197770    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"llm\"/\"premium-simulated-simulated-premium-kserve-self-signed-certs\""
May 06 17:40:24.208865 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.208840    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"]
May 06 17:40:24.235747 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.235718    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.235900 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.235778    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.235900 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.235811    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.235900 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.235840    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j2xkr\" (UniqueName: \"kubernetes.io/projected/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-kube-api-access-j2xkr\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.235900 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.235873    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.236087 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.235964    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.336658 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.336622    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.336856 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.336680    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.336856 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.336708    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.336856 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.336728    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.336856 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.336748    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j2xkr\" (UniqueName: \"kubernetes.io/projected/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-kube-api-access-j2xkr\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.336856 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.336769    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.337131 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.337083    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kserve-provision-location\" (UniqueName: \"kubernetes.io/empty-dir/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-kserve-provision-location\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.337230 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.337179    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"model-cache\" (UniqueName: \"kubernetes.io/empty-dir/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-model-cache\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.337230 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.337197    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"home\" (UniqueName: \"kubernetes.io/empty-dir/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-home\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.339017 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.338993    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dshm\" (UniqueName: \"kubernetes.io/empty-dir/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-dshm\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.339290 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.339270    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-certs\" (UniqueName: \"kubernetes.io/secret/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-tls-certs\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.345894 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.345870    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j2xkr\" (UniqueName: \"kubernetes.io/projected/ae111be5-f6c1-4d55-ba39-c3e9bb6b5787-kube-api-access-j2xkr\") pod \"premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn\" (UID: \"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787\") " pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.503970 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.503896    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:24.650028 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.649994    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"]
May 06 17:40:24.653306 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:40:24.653276    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podae111be5_f6c1_4d55_ba39_c3e9bb6b5787.slice/crio-3a23ee1097d772aca42897d445afce764eb907db682d2f1aa4fbbf9be762bb44 WatchSource:0}: Error finding container 3a23ee1097d772aca42897d445afce764eb907db682d2f1aa4fbbf9be762bb44: Status 404 returned error can't find the container with id 3a23ee1097d772aca42897d445afce764eb907db682d2f1aa4fbbf9be762bb44
May 06 17:40:24.710115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:24.710091    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn" event={"ID":"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787","Type":"ContainerStarted","Data":"3a23ee1097d772aca42897d445afce764eb907db682d2f1aa4fbbf9be762bb44"}
May 06 17:40:25.715162 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:25.715114    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn" event={"ID":"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787","Type":"ContainerStarted","Data":"0d170c573e8eb313ac6a866be134d0a6d3d7793d6dc4b2debd412e80f5cc2818"}
May 06 17:40:29.708389 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:29.708326    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm"
May 06 17:40:29.890707 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:29.890648    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:40:29.891847 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:29.891823    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:40:32.486984 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:32.486928    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:40:32.488127 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:32.488093    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:40:34.750702 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:34.750671    2560 generic.go:358] "Generic (PLEG): container finished" podID="ae111be5-f6c1-4d55-ba39-c3e9bb6b5787" containerID="0d170c573e8eb313ac6a866be134d0a6d3d7793d6dc4b2debd412e80f5cc2818" exitCode=0
May 06 17:40:34.751066 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:34.750741    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn" event={"ID":"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787","Type":"ContainerDied","Data":"0d170c573e8eb313ac6a866be134d0a6d3d7793d6dc4b2debd412e80f5cc2818"}
May 06 17:40:35.756264 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:35.756225    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn" event={"ID":"ae111be5-f6c1-4d55-ba39-c3e9bb6b5787","Type":"ContainerStarted","Data":"95e8de528659485587f1c1336776a973a4cf917447588da7c6bdbb31c1bb0e17"}
May 06 17:40:35.756639 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:35.756413    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:35.776895 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:35.776847    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn" podStartSLOduration=11.505065714 podStartE2EDuration="11.776833487s" podCreationTimestamp="2026-05-06 17:40:24 +0000 UTC" firstStartedPulling="2026-05-06 17:40:34.751378855 +0000 UTC m=+1806.195201279" lastFinishedPulling="2026-05-06 17:40:35.023146611 +0000 UTC m=+1806.466969052" observedRunningTime="2026-05-06 17:40:35.775133488 +0000 UTC m=+1807.218955936" watchObservedRunningTime="2026-05-06 17:40:35.776833487 +0000 UTC m=+1807.220655934"
May 06 17:40:42.639631 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:42.639575    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:40:42.640756 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:42.640731    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:40:45.594394 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:45.594337    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:40:45.595521 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:45.595498    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:40:46.778732 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:46.778699    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="llm/premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn"
May 06 17:40:56.117845 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:40:56.117803    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 06 17:40:56.613023 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:56.612969    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:40:56.614142 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:56.614120    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:40:57.443178 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:57.443105    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:40:57.444302 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:40:57.444278    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:41:12.413895 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:12.413809    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:41:12.415013 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:12.414988    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:41:12.589437 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:12.589378    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:41:12.590555 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:12.590535    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:41:25.379714 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:25.379661    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:41:25.380843 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:25.380819    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:41:28.574280 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:28.574222    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:41:28.575409 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:28.575385    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:41:38.565474 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:38.565417    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:41:38.566599 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:38.566577    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:41:40.445770 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:40.445710    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:41:40.446905 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:40.446878    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:41:51.446948 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:51.446893    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:41:51.448080 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:51.448057    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:41:53.384824 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:53.384767    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:41:53.385953 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:41:53.385928    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:42:04.461635 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:04.461552    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:42:04.462758 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:04.462732    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:42:07.491791 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:07.491739    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:42:07.492927 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:07.492901    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:42:18.432381 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:18.432313    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:42:18.433504 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:18.433479    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:42:20.387825 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:20.387771    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:42:20.388972 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:20.388936    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:42:31.383566 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:31.383509    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:42:31.384693 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:31.384669    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:42:31.420554 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:31.420496    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:42:31.421680 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:31.421655    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:42:43.355995 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:43.355926    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:42:43.357125 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:43.357101    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:42:43.766251 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:43.766180    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:42:43.767379 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:43.767353    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:42:55.475909 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:55.475840    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:42:55.477029 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:55.477003    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:42:58.389767 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:58.389709    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:42:58.391202 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:42:58.391173    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:43:07.575386 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:07.575329    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:43:07.576528 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:07.576505    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:43:13.385001 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:13.384949    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:43:13.386139 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:13.386117    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:43:22.471130 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:22.471072    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:43:22.472254 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:22.472232    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:43:28.510342 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:28.510281    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:43:28.511470 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:28.511443    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:43:33.480804 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:33.480751    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:43:33.481933 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:33.481911    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:43:39.479823 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:39.479750    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:43:39.480952 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:39.480928    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:43:46.507610 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:46.507539    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:43:46.508732 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:46.508707    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:43:53.390533 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:53.390472    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:43:53.391662 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:53.391636    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:43:59.496372 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:59.496318    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:43:59.498180 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:43:59.498140    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:44:05.374634 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:05.374563    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:44:05.375765 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:05.375740    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:44:10.572485 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:10.572413    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:44:10.573610 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:10.573588    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:44:18.399534 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:18.399480    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:44:18.400669 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:18.400645    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:44:25.493738 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:25.493683    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:44:25.494826 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:25.494805    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:44:32.371498 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:32.371423    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:44:32.372627 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:32.372603    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:44:38.507999 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:38.507943    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:44:38.509116 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:38.509098    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:44:44.457825 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:44.457762    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:44:44.458888 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:44.458864    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:44:49.526929 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:49.526872    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:44:49.528061 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:49.528037    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:44:57.406200 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:57.406125    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:44:57.407344 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:44:57.407297    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:45:04.515706 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:04.515649    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:45:04.516766 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:04.516735    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:45:10.448666 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:10.448606    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:45:10.449792 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:10.449767    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:45:19.456385 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:19.456330    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:45:19.457504 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:19.457484    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:45:26.424619 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:26.424565    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:45:26.425742 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:26.425716    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:45:33.502841 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:33.502770    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:45:33.503966 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:33.503942    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:45:41.437442 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:41.437365    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:45:41.438575 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:41.438547    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:45:44.445554 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:44.445497    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:45:44.446681 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:44.446657    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:45:54.602798 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:54.602738    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:45:54.603922 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:54.603899    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:45:55.458398 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:55.458346    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:45:55.459522 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:45:55.459499    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:46:07.117457 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:07.117419    2560 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 06 17:46:07.484754 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:07.484700    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:46:07.485876 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:07.485857    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:46:10.431852 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:10.431799    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:46:10.432987 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:10.432964    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:46:21.482363 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:21.482275    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:46:21.483494 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:21.483468    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:46:21.571292 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:21.571233    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:46:21.572410 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:21.572386    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:46:34.513276 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:34.513217    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:46:34.514436 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:34.514402    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:46:35.555477 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:35.555442    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67f5887d59-4c7xx_b78a113f-78e1-4197-a0d9-925e8bef3bd1/manager/0.log"
May 06 17:46:36.526699 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:36.526643    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:46:36.527818 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:36.527796    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:46:40.986502 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:40.986465    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67f5887d59-4c7xx_b78a113f-78e1-4197-a0d9-925e8bef3bd1/manager/0.log"
May 06 17:46:41.485487 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:41.485450    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-698574c4f-tnxwn_cfe6f3cf-2088-4a20-a377-a3eb27895bee/manager/0.log"
May 06 17:46:41.599752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:41.599721    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-868db5846d-jfqqx_18089601-8295-4947-b83e-02975d193a23/postgres/0.log"
May 06 17:46:42.385172 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:42.385129    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx_fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609/util/0.log"
May 06 17:46:42.392037 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:42.392014    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx_fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609/pull/0.log"
May 06 17:46:42.397892 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:42.397877    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx_fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609/extract/0.log"
May 06 17:46:42.522555 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:42.522525    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj_f268c192-5fa6-4fe2-82c0-16b8bf3f362d/util/0.log"
May 06 17:46:42.529449 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:42.529423    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj_f268c192-5fa6-4fe2-82c0-16b8bf3f362d/pull/0.log"
May 06 17:46:42.536067 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:42.536029    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj_f268c192-5fa6-4fe2-82c0-16b8bf3f362d/extract/0.log"
May 06 17:46:42.651186 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:42.651079    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt_81e66fce-71ac-43ed-9f81-101513c9741e/util/0.log"
May 06 17:46:42.658115 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:42.658096    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt_81e66fce-71ac-43ed-9f81-101513c9741e/pull/0.log"
May 06 17:46:42.665745 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:42.665726    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt_81e66fce-71ac-43ed-9f81-101513c9741e/extract/0.log"
May 06 17:46:42.778673 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:42.778638    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q_742300ba-658a-49b8-8af5-0cbac1d15924/pull/0.log"
May 06 17:46:42.784884 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:42.784865    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q_742300ba-658a-49b8-8af5-0cbac1d15924/extract/0.log"
May 06 17:46:42.790724 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:42.790699    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q_742300ba-658a-49b8-8af5-0cbac1d15924/util/0.log"
May 06 17:46:43.024435 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:43.024401    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-bbncm_e7f5efcb-bdc0-40d3-a2c0-dad2ad5133c7/manager/0.log"
May 06 17:46:43.251403 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:43.251375    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-gtpm4_80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f/kuadrant-console-plugin/0.log"
May 06 17:46:43.370209 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:43.370134    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-29x4m_2d7a32b4-3b9e-414f-bbac-a2492fa01c7c/registry-server/0.log"
May 06 17:46:44.100374 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:44.100339    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-595b7776f8swjrs_3171596d-a89d-4a55-bcfc-128408e618e0/istio-proxy/0.log"
May 06 17:46:44.574045 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:44.574007    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7df95f575-cjbmt_f23033c8-3765-45c5-af12-2a514f7179e8/istio-proxy/0.log"
May 06 17:46:45.073838 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:45.073797    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm_fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c/main/0.log"
May 06 17:46:45.081884 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:45.081863    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_e2e-distinct-2-simulated-kserve-7f849f6b56-kcssm_fcffb244-bd53-4fb0-b73b-b4bb6b2dc37c/storage-initializer/0.log"
May 06 17:46:45.699582 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:45.699553    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn_ae111be5-f6c1-4d55-ba39-c3e9bb6b5787/storage-initializer/0.log"
May 06 17:46:45.707787 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:45.707758    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/llm_premium-simulated-simulated-premium-kserve-6b97b89985-r2jxn_ae111be5-f6c1-4d55-ba39-c3e9bb6b5787/main/0.log"
May 06 17:46:49.370386 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:49.370325    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:46:49.371531 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:49.371485    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:46:50.081864 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.081821    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xnzz7/must-gather-jqnjn"]
May 06 17:46:50.085464 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.085441    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xnzz7/must-gather-jqnjn"
May 06 17:46:50.087657 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.087636    2560 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-xnzz7\"/\"default-dockercfg-crpl7\""
May 06 17:46:50.087773 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.087653    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xnzz7\"/\"kube-root-ca.crt\""
May 06 17:46:50.087773 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.087643    2560 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-xnzz7\"/\"openshift-service-ca.crt\""
May 06 17:46:50.101707 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.101680    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xnzz7/must-gather-jqnjn"]
May 06 17:46:50.166364 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.166325    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2a461142-2af5-434b-9a68-0e51bd2dba42-must-gather-output\") pod \"must-gather-jqnjn\" (UID: \"2a461142-2af5-434b-9a68-0e51bd2dba42\") " pod="openshift-must-gather-xnzz7/must-gather-jqnjn"
May 06 17:46:50.166620 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.166591    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ts2rp\" (UniqueName: \"kubernetes.io/projected/2a461142-2af5-434b-9a68-0e51bd2dba42-kube-api-access-ts2rp\") pod \"must-gather-jqnjn\" (UID: \"2a461142-2af5-434b-9a68-0e51bd2dba42\") " pod="openshift-must-gather-xnzz7/must-gather-jqnjn"
May 06 17:46:50.267805 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.267746    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2a461142-2af5-434b-9a68-0e51bd2dba42-must-gather-output\") pod \"must-gather-jqnjn\" (UID: \"2a461142-2af5-434b-9a68-0e51bd2dba42\") " pod="openshift-must-gather-xnzz7/must-gather-jqnjn"
May 06 17:46:50.267805 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.267806    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ts2rp\" (UniqueName: \"kubernetes.io/projected/2a461142-2af5-434b-9a68-0e51bd2dba42-kube-api-access-ts2rp\") pod \"must-gather-jqnjn\" (UID: \"2a461142-2af5-434b-9a68-0e51bd2dba42\") " pod="openshift-must-gather-xnzz7/must-gather-jqnjn"
May 06 17:46:50.268100 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.268076    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/2a461142-2af5-434b-9a68-0e51bd2dba42-must-gather-output\") pod \"must-gather-jqnjn\" (UID: \"2a461142-2af5-434b-9a68-0e51bd2dba42\") " pod="openshift-must-gather-xnzz7/must-gather-jqnjn"
May 06 17:46:50.275879 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.275847    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ts2rp\" (UniqueName: \"kubernetes.io/projected/2a461142-2af5-434b-9a68-0e51bd2dba42-kube-api-access-ts2rp\") pod \"must-gather-jqnjn\" (UID: \"2a461142-2af5-434b-9a68-0e51bd2dba42\") " pod="openshift-must-gather-xnzz7/must-gather-jqnjn"
May 06 17:46:50.395776 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.395677    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xnzz7/must-gather-jqnjn"
May 06 17:46:50.533350 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:50.533322    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xnzz7/must-gather-jqnjn"]
May 06 17:46:50.536048 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:46:50.536017    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod2a461142_2af5_434b_9a68_0e51bd2dba42.slice/crio-3eb9501630a132787246031bf24faea04cb5e5bcccf86576ffe4b3bc39b50753 WatchSource:0}: Error finding container 3eb9501630a132787246031bf24faea04cb5e5bcccf86576ffe4b3bc39b50753: Status 404 returned error can't find the container with id 3eb9501630a132787246031bf24faea04cb5e5bcccf86576ffe4b3bc39b50753
May 06 17:46:51.218026 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:51.217962    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xnzz7/must-gather-jqnjn" event={"ID":"2a461142-2af5-434b-9a68-0e51bd2dba42","Type":"ContainerStarted","Data":"3eb9501630a132787246031bf24faea04cb5e5bcccf86576ffe4b3bc39b50753"}
May 06 17:46:51.572945 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:51.572682    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:46:51.574554 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:46:51.574522    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:46:57.246950 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:57.246904    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xnzz7/must-gather-jqnjn" event={"ID":"2a461142-2af5-434b-9a68-0e51bd2dba42","Type":"ContainerStarted","Data":"d19eeb72389389ef70fdccb1b055c581adf7346d0c49c9cdd2ecb0eb9a7cc565"}
May 06 17:46:57.247597 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:57.247572    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xnzz7/must-gather-jqnjn" event={"ID":"2a461142-2af5-434b-9a68-0e51bd2dba42","Type":"ContainerStarted","Data":"3ad19bd941f2c1dbd7294128dc08d6463653f2226a9f07f692f3bbd07db43df4"}
May 06 17:46:57.268349 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:57.268282    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-xnzz7/must-gather-jqnjn" podStartSLOduration=1.60210765 podStartE2EDuration="7.26826025s" podCreationTimestamp="2026-05-06 17:46:50 +0000 UTC" firstStartedPulling="2026-05-06 17:46:50.537745803 +0000 UTC m=+2181.981568228" lastFinishedPulling="2026-05-06 17:46:56.203898389 +0000 UTC m=+2187.647720828" observedRunningTime="2026-05-06 17:46:57.264580196 +0000 UTC m=+2188.708402644" watchObservedRunningTime="2026-05-06 17:46:57.26826025 +0000 UTC m=+2188.712082698"
May 06 17:46:57.913309 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:57.913270    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6jh2w_3c0cf9e2-df1e-4740-89a9-5a9830ceb6d9/global-pull-secret-syncer/0.log"
May 06 17:46:58.114710 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:58.114584    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-sdndx_e360f9ba-379a-45f3-9ae6-dee0e26375db/konnectivity-agent/0.log"
May 06 17:46:58.138918 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:46:58.138884    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-130-186.ec2.internal_a35c20fbb6c420ebcfcdda7c5c935059/haproxy/0.log"
May 06 17:47:00.569176 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:47:00.569013    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:47:00.571851 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:47:00.571448    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:47:01.945144 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:01.945086    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx_fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609/extract/0.log"
May 06 17:47:01.971031 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:01.970995    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx_fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609/util/0.log"
May 06 17:47:01.997600 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:01.997554    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_0acee64185f523d1d1272e9af2e4d9333e0dcde792ba30e1fa9605b759dcdbx_fac5f175-4e7d-4e1b-a7d3-2cc92dcf0609/pull/0.log"
May 06 17:47:02.040076 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:02.040039    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj_f268c192-5fa6-4fe2-82c0-16b8bf3f362d/extract/0.log"
May 06 17:47:02.062946 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:02.062840    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj_f268c192-5fa6-4fe2-82c0-16b8bf3f362d/util/0.log"
May 06 17:47:02.088777 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:02.088747    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_19cb86e64775c5699d5aacf881a09c2d51e7f55f9e1ff096f2a667c5e0gzqdj_f268c192-5fa6-4fe2-82c0-16b8bf3f362d/pull/0.log"
May 06 17:47:02.118026 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:02.117979    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt_81e66fce-71ac-43ed-9f81-101513c9741e/extract/0.log"
May 06 17:47:02.141042 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:02.141004    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt_81e66fce-71ac-43ed-9f81-101513c9741e/util/0.log"
May 06 17:47:02.166963 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:02.166934    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_5fc36eb1065777914bfe0ff85f9a202e2a3bafefb563d9e2994474ed73mz2nt_81e66fce-71ac-43ed-9f81-101513c9741e/pull/0.log"
May 06 17:47:02.193268 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:02.193237    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q_742300ba-658a-49b8-8af5-0cbac1d15924/extract/0.log"
May 06 17:47:02.217130 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:02.217055    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q_742300ba-658a-49b8-8af5-0cbac1d15924/util/0.log"
May 06 17:47:02.244070 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:02.244032    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_9438e18bbad664a016701a3153a6b421e2d977be7eb0117f80bcf45ef1v8t2q_742300ba-658a-49b8-8af5-0cbac1d15924/pull/0.log"
May 06 17:47:02.305776 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:02.305731    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_authorino-operator-657f44b778-bbncm_e7f5efcb-bdc0-40d3-a2c0-dad2ad5133c7/manager/0.log"
May 06 17:47:02.354031 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:02.354001    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-console-plugin-6cb54b5c86-gtpm4_80ea41c0-c4c0-4fc7-9fc1-2d510a71f14f/kuadrant-console-plugin/0.log"
May 06 17:47:02.398764 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:02.398734    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/kuadrant-system_kuadrant-operator-catalog-29x4m_2d7a32b4-3b9e-414f-bbac-a2492fa01c7c/registry-server/0.log"
May 06 17:47:03.873484 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:03.873437    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7764dcf94f-rkpbs_f99cc8a8-6961-4cd2-b612-920c74b2b4a3/kube-state-metrics/0.log"
May 06 17:47:03.896645 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:03.896605    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7764dcf94f-rkpbs_f99cc8a8-6961-4cd2-b612-920c74b2b4a3/kube-rbac-proxy-main/0.log"
May 06 17:47:03.918847 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:03.918810    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-7764dcf94f-rkpbs_f99cc8a8-6961-4cd2-b612-920c74b2b4a3/kube-rbac-proxy-self/0.log"
May 06 17:47:04.182526 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:04.182270    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-msrpm_e5bf6bb6-c650-4f3a-996d-0cfe4d87178a/node-exporter/0.log"
May 06 17:47:04.206824 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:04.206741    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-msrpm_e5bf6bb6-c650-4f3a-996d-0cfe4d87178a/kube-rbac-proxy/0.log"
May 06 17:47:04.245263 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:04.245225    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-msrpm_e5bf6bb6-c650-4f3a-996d-0cfe4d87178a/init-textfile/0.log"
May 06 17:47:04.537448 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:04.537418    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-94789f4d5-f4kwf_fb3c196b-d702-4591-a31b-a48654c28dd5/prometheus-operator/0.log"
May 06 17:47:04.569317 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:04.569246    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_prometheus-operator-94789f4d5-f4kwf_fb3c196b-d702-4591-a31b-a48654c28dd5/kube-rbac-proxy/0.log"
May 06 17:47:04.706469 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:04.706434    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d65767784-9t7ms_c32ccae0-4d99-4a68-b85a-2406d8b6f16a/thanos-query/0.log"
May 06 17:47:04.732326 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:04.732266    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d65767784-9t7ms_c32ccae0-4d99-4a68-b85a-2406d8b6f16a/kube-rbac-proxy-web/0.log"
May 06 17:47:04.754391 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:04.754365    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d65767784-9t7ms_c32ccae0-4d99-4a68-b85a-2406d8b6f16a/kube-rbac-proxy/0.log"
May 06 17:47:04.779640 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:04.779608    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d65767784-9t7ms_c32ccae0-4d99-4a68-b85a-2406d8b6f16a/prom-label-proxy/0.log"
May 06 17:47:04.811230 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:04.811134    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d65767784-9t7ms_c32ccae0-4d99-4a68-b85a-2406d8b6f16a/kube-rbac-proxy-rules/0.log"
May 06 17:47:04.851129 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:04.851092    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-6d65767784-9t7ms_c32ccae0-4d99-4a68-b85a-2406d8b6f16a/kube-rbac-proxy-metrics/0.log"
May 06 17:47:05.628259 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:47:05.627859    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:47:05.630393 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:47:05.630362    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:47:06.318314 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.318274    2560 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"]
May 06 17:47:06.324726 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.324687    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.329682 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.329655    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"]
May 06 17:47:06.439883 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.439840    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/716dfb29-e48b-4f16-8e22-0797f47adc59-sys\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.440221 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.440192    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/716dfb29-e48b-4f16-8e22-0797f47adc59-lib-modules\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.440368 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.440337    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/716dfb29-e48b-4f16-8e22-0797f47adc59-podres\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.440474 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.440413    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6jc9f\" (UniqueName: \"kubernetes.io/projected/716dfb29-e48b-4f16-8e22-0797f47adc59-kube-api-access-6jc9f\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.440474 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.440439    2560 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/716dfb29-e48b-4f16-8e22-0797f47adc59-proc\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.541294 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.541252    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/716dfb29-e48b-4f16-8e22-0797f47adc59-sys\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.541294 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.541301    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/716dfb29-e48b-4f16-8e22-0797f47adc59-lib-modules\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.541550 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.541326    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/716dfb29-e48b-4f16-8e22-0797f47adc59-podres\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.541550 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.541360    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6jc9f\" (UniqueName: \"kubernetes.io/projected/716dfb29-e48b-4f16-8e22-0797f47adc59-kube-api-access-6jc9f\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.541550 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.541379    2560 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/716dfb29-e48b-4f16-8e22-0797f47adc59-proc\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.541550 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.541377    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/716dfb29-e48b-4f16-8e22-0797f47adc59-sys\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.541550 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.541452    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/716dfb29-e48b-4f16-8e22-0797f47adc59-proc\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.541550 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.541490    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/716dfb29-e48b-4f16-8e22-0797f47adc59-podres\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.541550 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.541543    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/716dfb29-e48b-4f16-8e22-0797f47adc59-lib-modules\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.550358 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.550306    2560 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6jc9f\" (UniqueName: \"kubernetes.io/projected/716dfb29-e48b-4f16-8e22-0797f47adc59-kube-api-access-6jc9f\") pod \"perf-node-gather-daemonset-xpx4c\" (UID: \"716dfb29-e48b-4f16-8e22-0797f47adc59\") " pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.639939 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.639320    2560 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:06.789350 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:06.789322    2560 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"]
May 06 17:47:06.792084 ip-10-0-130-186 kubenswrapper[2560]: W0506 17:47:06.792055    2560 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod716dfb29_e48b_4f16_8e22_0797f47adc59.slice/crio-19f5443a3748a715c33fa173c46ef9ffc877a05532cd1560756d42588cb70158 WatchSource:0}: Error finding container 19f5443a3748a715c33fa173c46ef9ffc877a05532cd1560756d42588cb70158: Status 404 returned error can't find the container with id 19f5443a3748a715c33fa173c46ef9ffc877a05532cd1560756d42588cb70158
May 06 17:47:07.053806 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:07.053779    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-c8c4474bc-h4vnh_50600190-c8d0-4091-ad42-1655c5bb4763/download-server/0.log"
May 06 17:47:07.307258 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:07.307222    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c" event={"ID":"716dfb29-e48b-4f16-8e22-0797f47adc59","Type":"ContainerStarted","Data":"af628b4f9f97d4f724b1a8e4306ce5d4c7c06ace44bcdd0659312dbcf47cdff7"}
May 06 17:47:07.307258 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:07.307261    2560 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c" event={"ID":"716dfb29-e48b-4f16-8e22-0797f47adc59","Type":"ContainerStarted","Data":"19f5443a3748a715c33fa173c46ef9ffc877a05532cd1560756d42588cb70158"}
May 06 17:47:07.307516 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:07.307365    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:07.324358 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:07.324314    2560 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c" podStartSLOduration=1.324298489 podStartE2EDuration="1.324298489s" podCreationTimestamp="2026-05-06 17:47:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-06 17:47:07.322593859 +0000 UTC m=+2198.766416303" watchObservedRunningTime="2026-05-06 17:47:07.324298489 +0000 UTC m=+2198.768120936"
May 06 17:47:08.570226 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:08.570182    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-thrt5_bf1db626-38d5-434d-a7ec-d48493c2605b/dns/0.log"
May 06 17:47:08.608488 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:08.608455    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-thrt5_bf1db626-38d5-434d-a7ec-d48493c2605b/kube-rbac-proxy/0.log"
May 06 17:47:08.717108 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:08.717082    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-4nq4b_3378859b-bdd9-4a9f-98cf-dba6a953b3f7/dns-node-resolver/0.log"
May 06 17:47:09.249206 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:09.249145    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-5b6fb99794-56z5g_e1921d6f-0f8e-4e1e-8207-e305837a82da/registry/0.log"
May 06 17:47:09.318747 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:09.318717    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-tfv78_61680c01-bda6-4b45-ae5b-32adc5ce27c8/node-ca/0.log"
May 06 17:47:10.264055 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:10.264022    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_data-science-gateway-data-science-gateway-class-595b7776f8swjrs_3171596d-a89d-4a55-bcfc-128408e618e0/istio-proxy/0.log"
May 06 17:47:10.402765 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:10.402739    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_maas-default-gateway-openshift-default-7df95f575-cjbmt_f23033c8-3765-45c5-af12-2a514f7179e8/istio-proxy/0.log"
May 06 17:47:10.947870 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:10.947840    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-2b4zh_995b0633-d889-4d0e-ab20-1ab11cef7e8d/serve-healthcheck-canary/0.log"
May 06 17:47:11.540054 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:11.540022    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-866jg_d19c0060-6e95-4256-a8db-eeebee71e3a6/kube-rbac-proxy/0.log"
May 06 17:47:11.563366 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:11.563323    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-866jg_d19c0060-6e95-4256-a8db-eeebee71e3a6/exporter/0.log"
May 06 17:47:11.585969 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:11.585939    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-866jg_d19c0060-6e95-4256-a8db-eeebee71e3a6/extractor/0.log"
May 06 17:47:13.322934 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:13.322905    2560 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-xnzz7/perf-node-gather-daemonset-xpx4c"
May 06 17:47:13.614790 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:13.614717    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_maas-controller-67f5887d59-4c7xx_b78a113f-78e1-4197-a0d9-925e8bef3bd1/manager/0.log"
May 06 17:47:13.729335 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:13.729306    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_opendatahub-operator-controller-manager-698574c4f-tnxwn_cfe6f3cf-2088-4a20-a377-a3eb27895bee/manager/0.log"
May 06 17:47:13.758029 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:13.758003    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/opendatahub_postgres-868db5846d-jfqqx_18089601-8295-4947-b83e-02975d193a23/postgres/0.log"
May 06 17:47:14.443880 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:47:14.443798    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api:latest,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-h4psz,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-7f4595bb66-9kzqf_opendatahub(481d4898-bc40-46ab-8131-f09a9cb71be4): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:47:14.445039 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:47:14.445005    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-7f4595bb66-9kzqf" podUID="481d4898-bc40-46ab-8131-f09a9cb71be4"
May 06 17:47:15.022615 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:15.022540    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-lws-operator_lws-controller-manager-7fd7474bdd-dsz2k_3a78bdda-da4e-4f60-8738-055fef49eecb/manager/0.log"
May 06 17:47:17.471667 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:47:17.471270    2560 kuberuntime_manager.go:1358] "Unhandled Error" err="container &Container{Name:maas-api,Image:quay.io/opendatahub/maas-api@sha256:02888aac23b187629ab160232d6f00b17570cd7d9247e72c24c9e7f3a233f6af,Command:[],Args:[],WorkingDir:,Ports:[]ContainerPort{ContainerPort{Name:https,HostPort:0,ContainerPort:8443,Protocol:TCP,HostIP:,},},Env:[]EnvVar{EnvVar{Name:GATEWAY_NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-namespace,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:GATEWAY_NAME,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:gateway-name,Optional:nil,},SecretKeyRef:nil,},},EnvVar{Name:API_KEY_MAX_EXPIRATION_DAYS,Value:,ValueFrom:&EnvVarSource{FieldRef:nil,ResourceFieldRef:nil,ConfigMapKeyRef:&ConfigMapKeySelector{LocalObjectReference:LocalObjectReference{Name:maas-parameters,},Key:api-key-max-expiration-days,Optional:*true,},SecretKeyRef:nil,},},EnvVar{Name:SECURE,Value:true,ValueFrom:nil,},EnvVar{Name:TLS_CERT,Value:/etc/maas-api/tls/tls.crt,ValueFrom:nil,},EnvVar{Name:TLS_KEY,Value:/etc/maas-api/tls/tls.key,ValueFrom:nil,},EnvVar{Name:NAMESPACE,Value:,ValueFrom:&EnvVarSource{FieldRef:&ObjectFieldSelector{APIVersion:v1,FieldPath:metadata.namespace,},ResourceFieldRef:nil,ConfigMapKeyRef:nil,SecretKeyRef:nil,},},EnvVar{Name:MAAS_SUBSCRIPTION_NAMESPACE,Value:models-as-a-service,ValueFrom:nil,},EnvVar{Name:METRICS_PORT,Value:9090,ValueFrom:nil,},},Resources:ResourceRequirements{Limits:ResourceList{cpu: {{200 -3} {<nil>} 200m DecimalSI},memory: {{134217728 0} {<nil>}  BinarySI},},Requests:ResourceList{cpu: {{50 -3} {<nil>} 50m DecimalSI},memory: {{67108864 0} {<nil>}  BinarySI},},Claims:[]ResourceClaim{},},VolumeMounts:[]VolumeMount{VolumeMount{Name:maas-api-tls,ReadOnly:true,MountPath:/etc/maas-api/tls,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},VolumeMount{Name:kube-api-access-zzx46,ReadOnly:true,MountPath:/var/run/secrets/kubernetes.io/serviceaccount,SubPath:,MountPropagation:nil,SubPathExpr:,RecursiveReadOnly:nil,},},LivenessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:30,TimeoutSeconds:5,PeriodSeconds:10,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},ReadinessProbe:&Probe{ProbeHandler:ProbeHandler{Exec:nil,HTTPGet:&HTTPGetAction{Path:/health,Port:{1 0 https},Host:,Scheme:HTTPS,HTTPHeaders:[]HTTPHeader{},},TCPSocket:nil,GRPC:nil,},InitialDelaySeconds:5,TimeoutSeconds:3,PeriodSeconds:5,SuccessThreshold:1,FailureThreshold:3,TerminationGracePeriodSeconds:nil,},Lifecycle:nil,TerminationMessagePath:/dev/termination-log,ImagePullPolicy:Always,SecurityContext:&SecurityContext{Capabilities:&Capabilities{Add:[],Drop:[ALL],},Privileged:nil,SELinuxOptions:nil,RunAsUser:*1000660000,RunAsNonRoot:*true,ReadOnlyRootFilesystem:*true,AllowPrivilegeEscalation:*false,RunAsGroup:nil,ProcMount:nil,WindowsOptions:nil,SeccompProfile:nil,AppArmorProfile:nil,},Stdin:false,StdinOnce:false,TTY:false,EnvFrom:[]EnvFromSource{},TerminationMessagePolicy:File,VolumeDevices:[]VolumeDevice{},StartupProbe:nil,ResizePolicy:[]ContainerResizePolicy{},RestartPolicy:nil,} start failed in pod maas-api-5c87f7f96f-dmssh_opendatahub(de6f9aea-3484-4180-9db2-82cbf9602ce9): CreateContainerConfigError: configmap \"maas-parameters\" not found" logger="UnhandledError"
May 06 17:47:17.473190 ip-10-0-130-186 kubenswrapper[2560]: E0506 17:47:17.473131    2560 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"maas-api\" with CreateContainerConfigError: \"configmap \\\"maas-parameters\\\" not found\"" pod="opendatahub/maas-api-5c87f7f96f-dmssh" podUID="de6f9aea-3484-4180-9db2-82cbf9602ce9"
May 06 17:47:21.031243 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:21.031210    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-66rrf_bde1dd92-05d2-474d-b20d-0bf6c96d8cfd/kube-multus/0.log"
May 06 17:47:21.239878 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:21.239848    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-klb5m_8feb5920-e85e-4a4d-bcbd-52a8369c0eec/kube-multus-additional-cni-plugins/0.log"
May 06 17:47:21.261752 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:21.261724    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-klb5m_8feb5920-e85e-4a4d-bcbd-52a8369c0eec/egress-router-binary-copy/0.log"
May 06 17:47:21.285405 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:21.285318    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-klb5m_8feb5920-e85e-4a4d-bcbd-52a8369c0eec/cni-plugins/0.log"
May 06 17:47:21.307211 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:21.307182    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-klb5m_8feb5920-e85e-4a4d-bcbd-52a8369c0eec/bond-cni-plugin/0.log"
May 06 17:47:21.330483 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:21.330409    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-klb5m_8feb5920-e85e-4a4d-bcbd-52a8369c0eec/routeoverride-cni/0.log"
May 06 17:47:21.353474 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:21.353445    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-klb5m_8feb5920-e85e-4a4d-bcbd-52a8369c0eec/whereabouts-cni-bincopy/0.log"
May 06 17:47:21.379296 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:21.379267    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-klb5m_8feb5920-e85e-4a4d-bcbd-52a8369c0eec/whereabouts-cni/0.log"
May 06 17:47:21.682110 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:21.682078    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cwlz9_24a44446-b589-49fd-a785-92e01601b8b4/network-metrics-daemon/0.log"
May 06 17:47:21.734704 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:21.734675    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-cwlz9_24a44446-b589-49fd-a785-92e01601b8b4/kube-rbac-proxy/0.log"
May 06 17:47:22.764820 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:22.764793    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9s6rp_b28d0114-0ff1-4b91-873b-467eb3f73088/ovn-controller/0.log"
May 06 17:47:22.810285 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:22.810257    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9s6rp_b28d0114-0ff1-4b91-873b-467eb3f73088/ovn-acl-logging/0.log"
May 06 17:47:22.827329 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:22.827300    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9s6rp_b28d0114-0ff1-4b91-873b-467eb3f73088/kube-rbac-proxy-node/0.log"
May 06 17:47:22.848333 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:22.848307    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9s6rp_b28d0114-0ff1-4b91-873b-467eb3f73088/kube-rbac-proxy-ovn-metrics/0.log"
May 06 17:47:22.868898 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:22.868863    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9s6rp_b28d0114-0ff1-4b91-873b-467eb3f73088/northd/0.log"
May 06 17:47:22.890108 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:22.890049    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9s6rp_b28d0114-0ff1-4b91-873b-467eb3f73088/nbdb/0.log"
May 06 17:47:22.912916 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:22.912895    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9s6rp_b28d0114-0ff1-4b91-873b-467eb3f73088/sbdb/0.log"
May 06 17:47:23.049583 ip-10-0-130-186 kubenswrapper[2560]: I0506 17:47:23.049550    2560 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-9s6rp_b28d0114-0ff1-4b91-873b-467eb3f73088/ovnkube-controller/0.log"