May 25 06:46:01.655161 ip-10-0-128-6 systemd[1]: Starting Kubernetes Kubelet...
May 25 06:46:02.075062 ip-10-0-128-6 kubenswrapper[2572]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 25 06:46:02.075062 ip-10-0-128-6 kubenswrapper[2572]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 25 06:46:02.075062 ip-10-0-128-6 kubenswrapper[2572]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 25 06:46:02.075062 ip-10-0-128-6 kubenswrapper[2572]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 25 06:46:02.075062 ip-10-0-128-6 kubenswrapper[2572]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 25 06:46:02.076625 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.076474    2572 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 25 06:46:02.079279 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079258    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 06:46:02.079279 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079275    2572 feature_gate.go:328] unrecognized feature gate: Example2
May 25 06:46:02.079279 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079280    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 06:46:02.079279 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079287    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079292    2572 feature_gate.go:328] unrecognized feature gate: Example
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079296    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079300    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079303    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079309    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079320    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079324    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079328    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079332    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079336    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079339    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079343    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079347    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079351    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079356    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079359    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079363    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079367    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 06:46:02.079526 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079371    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079375    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079379    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079383    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079386    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079391    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079395    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079399    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079402    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079407    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079411    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079416    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079420    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079424    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079429    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079435    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079440    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079444    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079448    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079453    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 06:46:02.080347 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079457    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079461    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079465    2572 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079469    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079474    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079478    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079482    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079486    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079491    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079495    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079499    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079503    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079507    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079512    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079516    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079520    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079525    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079529    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079535    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079540    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 06:46:02.081043 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079544    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079549    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079553    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079557    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079563    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079567    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079571    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079577    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079582    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079588    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079594    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079599    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079606    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079610    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079615    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079620    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079624    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079629    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079633    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 06:46:02.081777 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079638    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079642    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079646    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079650    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.079654    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080289    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080299    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080303    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080307    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080312    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080315    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080320    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080324    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080328    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080332    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080336    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080340    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080345    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080349    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080353    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 06:46:02.082602 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080358    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080362    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080366    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080370    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080374    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080378    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080382    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080386    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080390    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080395    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080400    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080404    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080409    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080413    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080417    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080421    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080425    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080429    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080434    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 06:46:02.083206 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080438    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080442    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080446    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080450    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080454    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080458    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080462    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080466    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080470    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080474    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080478    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080483    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080488    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080493    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080497    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080503    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080507    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080511    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080515    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080519    2572 feature_gate.go:328] unrecognized feature gate: Example
May 25 06:46:02.083773 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080523    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080527    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080532    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080536    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080540    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080544    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080548    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080553    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080557    2572 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080561    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080565    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080568    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080573    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080577    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080581    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080586    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080590    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080594    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080598    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080602    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 06:46:02.084315 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080606    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080610    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080614    2572 feature_gate.go:328] unrecognized feature gate: Example2
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080621    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080629    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080636    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080641    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080646    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080651    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080655    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080660    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.080664    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081341    2572 flags.go:64] FLAG: --address="0.0.0.0"
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081356    2572 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081367    2572 flags.go:64] FLAG: --anonymous-auth="true"
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081374    2572 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081381    2572 flags.go:64] FLAG: --authentication-token-webhook="false"
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081386    2572 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081393    2572 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081400    2572 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081406    2572 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 25 06:46:02.084817 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081411    2572 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081417    2572 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081422    2572 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081428    2572 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081433    2572 flags.go:64] FLAG: --cgroup-root=""
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081438    2572 flags.go:64] FLAG: --cgroups-per-qos="true"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081442    2572 flags.go:64] FLAG: --client-ca-file=""
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081447    2572 flags.go:64] FLAG: --cloud-config=""
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081452    2572 flags.go:64] FLAG: --cloud-provider="external"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081457    2572 flags.go:64] FLAG: --cluster-dns="[]"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081463    2572 flags.go:64] FLAG: --cluster-domain=""
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081468    2572 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081473    2572 flags.go:64] FLAG: --config-dir=""
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081478    2572 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081484    2572 flags.go:64] FLAG: --container-log-max-files="5"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081492    2572 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081498    2572 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081504    2572 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081510    2572 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081515    2572 flags.go:64] FLAG: --contention-profiling="false"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081520    2572 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081525    2572 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081530    2572 flags.go:64] FLAG: --cpu-manager-policy="none"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081535    2572 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081542    2572 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 25 06:46:02.085361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081548    2572 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081552    2572 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081557    2572 flags.go:64] FLAG: --enable-load-reader="false"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081562    2572 flags.go:64] FLAG: --enable-server="true"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081567    2572 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081576    2572 flags.go:64] FLAG: --event-burst="100"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081581    2572 flags.go:64] FLAG: --event-qps="50"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081586    2572 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081591    2572 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081597    2572 flags.go:64] FLAG: --eviction-hard=""
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081604    2572 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081608    2572 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081614    2572 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081619    2572 flags.go:64] FLAG: --eviction-soft=""
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081624    2572 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081629    2572 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081634    2572 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081639    2572 flags.go:64] FLAG: --experimental-mounter-path=""
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081644    2572 flags.go:64] FLAG: --fail-cgroupv1="false"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081648    2572 flags.go:64] FLAG: --fail-swap-on="true"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081653    2572 flags.go:64] FLAG: --feature-gates=""
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081660    2572 flags.go:64] FLAG: --file-check-frequency="20s"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081665    2572 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081671    2572 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081677    2572 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081682    2572 flags.go:64] FLAG: --healthz-port="10248"
May 25 06:46:02.086004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081688    2572 flags.go:64] FLAG: --help="false"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081692    2572 flags.go:64] FLAG: --hostname-override="ip-10-0-128-6.ec2.internal"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081697    2572 flags.go:64] FLAG: --housekeeping-interval="10s"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081702    2572 flags.go:64] FLAG: --http-check-frequency="20s"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081708    2572 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081714    2572 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081720    2572 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081725    2572 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081730    2572 flags.go:64] FLAG: --image-service-endpoint=""
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081735    2572 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081740    2572 flags.go:64] FLAG: --kube-api-burst="100"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081745    2572 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081750    2572 flags.go:64] FLAG: --kube-api-qps="50"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081755    2572 flags.go:64] FLAG: --kube-reserved=""
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081760    2572 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081765    2572 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081771    2572 flags.go:64] FLAG: --kubelet-cgroups=""
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081775    2572 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081780    2572 flags.go:64] FLAG: --lock-file=""
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081784    2572 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081789    2572 flags.go:64] FLAG: --log-flush-frequency="5s"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081794    2572 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081802    2572 flags.go:64] FLAG: --log-json-split-stream="false"
May 25 06:46:02.086917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081807    2572 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081812    2572 flags.go:64] FLAG: --log-text-split-stream="false"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081818    2572 flags.go:64] FLAG: --logging-format="text"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081823    2572 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081828    2572 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081834    2572 flags.go:64] FLAG: --manifest-url=""
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081857    2572 flags.go:64] FLAG: --manifest-url-header=""
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081865    2572 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081871    2572 flags.go:64] FLAG: --max-open-files="1000000"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081879    2572 flags.go:64] FLAG: --max-pods="110"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081884    2572 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081889    2572 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081894    2572 flags.go:64] FLAG: --memory-manager-policy="None"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081899    2572 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081904    2572 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081908    2572 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081913    2572 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081925    2572 flags.go:64] FLAG: --node-status-max-images="50"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081930    2572 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081935    2572 flags.go:64] FLAG: --oom-score-adj="-999"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081940    2572 flags.go:64] FLAG: --pod-cidr=""
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081944    2572 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081953    2572 flags.go:64] FLAG: --pod-manifest-path=""
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081958    2572 flags.go:64] FLAG: --pod-max-pids="-1"
May 25 06:46:02.087508 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081963    2572 flags.go:64] FLAG: --pods-per-core="0"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081968    2572 flags.go:64] FLAG: --port="10250"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081972    2572 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081977    2572 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0bd82c4fd2dfb28a1"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081982    2572 flags.go:64] FLAG: --qos-reserved=""
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081987    2572 flags.go:64] FLAG: --read-only-port="10255"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081992    2572 flags.go:64] FLAG: --register-node="true"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.081997    2572 flags.go:64] FLAG: --register-schedulable="true"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082001    2572 flags.go:64] FLAG: --register-with-taints=""
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082007    2572 flags.go:64] FLAG: --registry-burst="10"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082012    2572 flags.go:64] FLAG: --registry-qps="5"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082016    2572 flags.go:64] FLAG: --reserved-cpus=""
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082021    2572 flags.go:64] FLAG: --reserved-memory=""
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082028    2572 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082039    2572 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082044    2572 flags.go:64] FLAG: --rotate-certificates="false"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082049    2572 flags.go:64] FLAG: --rotate-server-certificates="false"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082056    2572 flags.go:64] FLAG: --runonce="false"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082062    2572 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082068    2572 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082072    2572 flags.go:64] FLAG: --seccomp-default="false"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082077    2572 flags.go:64] FLAG: --serialize-image-pulls="true"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082082    2572 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082087    2572 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082092    2572 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082097    2572 flags.go:64] FLAG: --storage-driver-password="root"
May 25 06:46:02.088179 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082101    2572 flags.go:64] FLAG: --storage-driver-secure="false"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082106    2572 flags.go:64] FLAG: --storage-driver-table="stats"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082111    2572 flags.go:64] FLAG: --storage-driver-user="root"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082116    2572 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082121    2572 flags.go:64] FLAG: --sync-frequency="1m0s"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082126    2572 flags.go:64] FLAG: --system-cgroups=""
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082131    2572 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082140    2572 flags.go:64] FLAG: --system-reserved-cgroup=""
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082145    2572 flags.go:64] FLAG: --tls-cert-file=""
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082149    2572 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082156    2572 flags.go:64] FLAG: --tls-min-version=""
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082161    2572 flags.go:64] FLAG: --tls-private-key-file=""
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082165    2572 flags.go:64] FLAG: --topology-manager-policy="none"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082170    2572 flags.go:64] FLAG: --topology-manager-policy-options=""
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082175    2572 flags.go:64] FLAG: --topology-manager-scope="container"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082180    2572 flags.go:64] FLAG: --v="2"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082186    2572 flags.go:64] FLAG: --version="false"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082193    2572 flags.go:64] FLAG: --vmodule=""
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082199    2572 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.082205    2572 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082363    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082369    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082374    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082379    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 06:46:02.088791 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082386    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082391    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082403    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082408    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082412    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082417    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082421    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082425    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082429    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082433    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082437    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082441    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082445    2572 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082449    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082454    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082458    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082462    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082467    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082471    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 06:46:02.089414 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082475    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082479    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082483    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082487    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082492    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082496    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082500    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082504    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082508    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082514    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082518    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082522    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082526    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082533    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082538    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082542    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082548    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082555    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082559    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082563    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 06:46:02.089914 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082568    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082572    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082578    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082584    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082589    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082593    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082597    2572 feature_gate.go:328] unrecognized feature gate: Example2
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082601    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082606    2572 feature_gate.go:328] unrecognized feature gate: Example
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082609    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082613    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082617    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082621    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082625    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082630    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082634    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082638    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082642    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082646    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082651    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 06:46:02.090410 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082655    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082661    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082666    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082670    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082674    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082679    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082684    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082689    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082694    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082698    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082703    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082707    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082712    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082716    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082719    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082724    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082728    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082732    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082736    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082740    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 06:46:02.091015 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082745    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 06:46:02.091515 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082749    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 06:46:02.091515 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.082753    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 06:46:02.091515 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.083227    2572 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 25 06:46:02.092542 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.092522    2572 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 25 06:46:02.092577 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.092545    2572 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 25 06:46:02.092610 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092594    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 06:46:02.092610 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092600    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 06:46:02.092610 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092604    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 06:46:02.092610 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092607    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 06:46:02.092610 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092610    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 06:46:02.092610 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092613    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092616    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092619    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092621    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092624    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092627    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092631    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092636    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092639    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092642    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092645    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092648    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092651    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092654    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092657    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092659    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092662    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092665    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092667    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 06:46:02.092764 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092670    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092673    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092675    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092678    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092681    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092684    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092686    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092690    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092692    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092695    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092698    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092700    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092702    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092705    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092708    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092711    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092714    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092717    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092721    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092724    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 06:46:02.093252 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092727    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092730    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092733    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092735    2572 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092738    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092740    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092743    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092746    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092748    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092751    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092753    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092756    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092758    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092761    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092763    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092766    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092769    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092771    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092775    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092777    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 06:46:02.093733 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092780    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092783    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092786    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092789    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092791    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092794    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092797    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092800    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092803    2572 feature_gate.go:328] unrecognized feature gate: Example
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092806    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092808    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092811    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092814    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092816    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092819    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092821    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092824    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092826    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092829    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092831    2572 feature_gate.go:328] unrecognized feature gate: Example2
May 25 06:46:02.094226 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092834    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092836    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.092856    2572 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092954    2572 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092959    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092962    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092965    2572 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092968    2572 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092971    2572 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092974    2572 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092976    2572 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092979    2572 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092982    2572 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092985    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092988    2572 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 25 06:46:02.094715 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092991    2572 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092993    2572 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092996    2572 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.092999    2572 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093004    2572 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093009    2572 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093012    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093016    2572 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093019    2572 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093022    2572 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093024    2572 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093027    2572 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093029    2572 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093032    2572 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093034    2572 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093037    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093039    2572 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093042    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093044    2572 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 25 06:46:02.095104 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093047    2572 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093049    2572 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093051    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093054    2572 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093057    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093059    2572 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093062    2572 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093064    2572 feature_gate.go:328] unrecognized feature gate: NewOLM
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093067    2572 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093070    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093072    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093075    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093078    2572 feature_gate.go:328] unrecognized feature gate: Example2
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093081    2572 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093083    2572 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093086    2572 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093088    2572 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093091    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093093    2572 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093096    2572 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 25 06:46:02.095566 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093099    2572 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093102    2572 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093104    2572 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093107    2572 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093109    2572 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093112    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093115    2572 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093117    2572 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093120    2572 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093122    2572 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093125    2572 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093128    2572 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093130    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093133    2572 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093135    2572 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093138    2572 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093141    2572 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093143    2572 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093146    2572 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093148    2572 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 25 06:46:02.096077 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093151    2572 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093153    2572 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093156    2572 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093159    2572 feature_gate.go:328] unrecognized feature gate: Example
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093161    2572 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093165    2572 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093167    2572 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093170    2572 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093173    2572 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093175    2572 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093178    2572 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093181    2572 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093183    2572 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093186    2572 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:02.093188    2572 feature_gate.go:328] unrecognized feature gate: DualReplica
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.093194    2572 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 25 06:46:02.096562 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.093817    2572 server.go:962] "Client rotation is on, will bootstrap in background"
May 25 06:46:02.097057 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.095898    2572 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 25 06:46:02.097057 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.096780    2572 server.go:1019] "Starting client certificate rotation"
May 25 06:46:02.097057 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.096879    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 25 06:46:02.097560 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.097548    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 25 06:46:02.120433 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.120399    2572 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 25 06:46:02.126879 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.126854    2572 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 25 06:46:02.136388 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.136364    2572 log.go:25] "Validated CRI v1 runtime API"
May 25 06:46:02.141263 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.141246    2572 log.go:25] "Validated CRI v1 image API"
May 25 06:46:02.142620 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.142605    2572 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 25 06:46:02.145604 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.145577    2572 fs.go:135] Filesystem UUIDs: map[5ae1d11c-6fdf-438e-ade1-fb8e632d88f7:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 898b3e9a-acda-42bf-b7f2-647104bbba35:/dev/nvme0n1p4]
May 25 06:46:02.145676 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.145602    2572 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 25 06:46:02.149226 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.149206    2572 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 25 06:46:02.150739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.150618    2572 manager.go:217] Machine: {Timestamp:2026-05-25 06:46:02.150104848 +0000 UTC m=+0.386291231 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3101357 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec202fd5e3195f631a61d5ff599933bb SystemUUID:ec202fd5-e319-5f63-1a61-d5ff599933bb BootID:17bc91e7-2798-4242-be4e-14f844d476d0 Filesystems:[{Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5b:4f:fc:ce:b3 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5b:4f:fc:ce:b3 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:ce:20:be:c4:dc:1b Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 25 06:46:02.151864 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.151854    2572 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 25 06:46:02.151954 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.151941    2572 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 25 06:46:02.155334 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.155310    2572 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 25 06:46:02.155479 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.155338    2572 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-128-6.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 25 06:46:02.155522 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.155490    2572 topology_manager.go:138] "Creating topology manager with none policy"
May 25 06:46:02.155522 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.155499    2572 container_manager_linux.go:306] "Creating device plugin manager"
May 25 06:46:02.155522 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.155507    2572 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 25 06:46:02.156242 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.156231    2572 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 25 06:46:02.157049 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.157039    2572 state_mem.go:36] "Initialized new in-memory state store"
May 25 06:46:02.157164 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.157154    2572 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 25 06:46:02.159523 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.159513    2572 kubelet.go:491] "Attempting to sync node with API server"
May 25 06:46:02.159564 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.159526    2572 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 25 06:46:02.159564 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.159539    2572 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 25 06:46:02.159564 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.159548    2572 kubelet.go:397] "Adding apiserver pod source"
May 25 06:46:02.159564 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.159557    2572 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 25 06:46:02.160931 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.160917    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 25 06:46:02.160987 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.160940    2572 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 25 06:46:02.164603 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.164581    2572 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 25 06:46:02.167073 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.167043    2572 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 25 06:46:02.169447 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.169427    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 25 06:46:02.169507 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.169454    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 25 06:46:02.169507 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.169465    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 25 06:46:02.169507 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.169474    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 25 06:46:02.169507 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.169487    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 25 06:46:02.169507 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.169496    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 25 06:46:02.169507 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.169506    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 25 06:46:02.169674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.169515    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 25 06:46:02.169674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.169527    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 25 06:46:02.169674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.169536    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 25 06:46:02.169674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.169545    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 25 06:46:02.169674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.169551    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 25 06:46:02.170276 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.170263    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 25 06:46:02.170322 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.170282    2572 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 25 06:46:02.171599 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.171574    2572 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-128-6.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 25 06:46:02.171599 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.171576    2572 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 25 06:46:02.172513 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.172493    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-67bfj"
May 25 06:46:02.173667 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.173653    2572 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 25 06:46:02.173744 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.173694    2572 server.go:1295] "Started kubelet"
May 25 06:46:02.173796 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.173755    2572 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 25 06:46:02.173856 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.173782    2572 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 25 06:46:02.173911 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.173873    2572 server_v1.go:47] "podresources" method="list" useActivePods=true
May 25 06:46:02.174540 ip-10-0-128-6 systemd[1]: Started Kubernetes Kubelet.
May 25 06:46:02.175162 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.174944    2572 server.go:317] "Adding debug handlers to kubelet server"
May 25 06:46:02.175162 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.174953    2572 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 25 06:46:02.180112 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.180095    2572 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 25 06:46:02.180199 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.180182    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-67bfj"
May 25 06:46:02.180245 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.180205    2572 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 25 06:46:02.180860 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.180374    2572 volume_manager.go:295] "The desired_state_of_world populator starts"
May 25 06:46:02.180860 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.180428    2572 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 25 06:46:02.180860 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.180541    2572 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 25 06:46:02.180860 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.180617    2572 reconstruct.go:97] "Volume reconstruction finished"
May 25 06:46:02.180860 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.180626    2572 reconciler.go:26] "Reconciler: start to sync state"
May 25 06:46:02.181118 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.180949    2572 factory.go:55] Registering systemd factory
May 25 06:46:02.181118 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.180966    2572 factory.go:223] Registration of the systemd container factory successfully
May 25 06:46:02.181214 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.181194    2572 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-128-6.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 25 06:46:02.181264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.181224    2572 factory.go:153] Registering CRI-O factory
May 25 06:46:02.181264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.181239    2572 factory.go:223] Registration of the crio container factory successfully
May 25 06:46:02.181345 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.181306    2572 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 25 06:46:02.181345 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.181324    2572 factory.go:103] Registering Raw factory
May 25 06:46:02.181345 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.181332    2572 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-6.ec2.internal\" not found"
May 25 06:46:02.181490 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.181338    2572 manager.go:1196] Started watching for new ooms in manager
May 25 06:46:02.181836 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.181816    2572 manager.go:319] Starting recovery of all containers
May 25 06:46:02.185913 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.181245    2572 event.go:359] "Server rejected event (will not retry!)" err="events is forbidden: User \"system:anonymous\" cannot create resource \"events\" in API group \"\" in the namespace \"default\"" event="&Event{ObjectMeta:{ip-10-0-128-6.ec2.internal.18b2bc0bed1f1092  default    0 0001-01-01 00:00:00 +0000 UTC <nil> <nil> map[] map[] [] [] []},InvolvedObject:ObjectReference{Kind:Node,Namespace:,Name:ip-10-0-128-6.ec2.internal,UID:ip-10-0-128-6.ec2.internal,APIVersion:,ResourceVersion:,FieldPath:,},Reason:Starting,Message:Starting kubelet.,Source:EventSource{Component:kubelet,Host:ip-10-0-128-6.ec2.internal,},FirstTimestamp:2026-05-25 06:46:02.173665426 +0000 UTC m=+0.409851809,LastTimestamp:2026-05-25 06:46:02.173665426 +0000 UTC m=+0.409851809,Count:1,Type:Normal,EventTime:0001-01-01 00:00:00 +0000 UTC,Series:nil,Action:,Related:nil,ReportingController:kubelet,ReportingInstance:ip-10-0-128-6.ec2.internal,}"
May 25 06:46:02.187663 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.187638    2572 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 06:46:02.191080 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.191053    2572 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-128-6.ec2.internal\" not found" node="ip-10-0-128-6.ec2.internal"
May 25 06:46:02.194217 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.194195    2572 manager.go:324] Recovery completed
May 25 06:46:02.196058 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.196038    2572 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
May 25 06:46:02.199066 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.199055    2572 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 06:46:02.200911 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.200894    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasSufficientMemory"
May 25 06:46:02.200985 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.200923    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasNoDiskPressure"
May 25 06:46:02.200985 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.200933    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasSufficientPID"
May 25 06:46:02.201439 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.201420    2572 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 25 06:46:02.201501 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.201438    2572 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 25 06:46:02.201501 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.201459    2572 state_mem.go:36] "Initialized new in-memory state store"
May 25 06:46:02.203739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.203726    2572 policy_none.go:49] "None policy: Start"
May 25 06:46:02.203787 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.203743    2572 memory_manager.go:186] "Starting memorymanager" policy="None"
May 25 06:46:02.203787 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.203755    2572 state_mem.go:35] "Initializing new in-memory state store"
May 25 06:46:02.244633 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.244455    2572 manager.go:341] "Starting Device Plugin manager"
May 25 06:46:02.256265 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.244673    2572 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 25 06:46:02.256265 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.244690    2572 server.go:85] "Starting device plugin registration server"
May 25 06:46:02.256265 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.244974    2572 eviction_manager.go:189] "Eviction manager: starting control loop"
May 25 06:46:02.256265 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.244994    2572 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 25 06:46:02.256265 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.245086    2572 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 25 06:46:02.256265 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.245152    2572 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 25 06:46:02.256265 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.245161    2572 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 25 06:46:02.256265 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.245684    2572 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 25 06:46:02.256265 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.245722    2572 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-128-6.ec2.internal\" not found"
May 25 06:46:02.276312 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.276274    2572 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 25 06:46:02.277530 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.277514    2572 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 25 06:46:02.277599 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.277538    2572 status_manager.go:230] "Starting to sync pod status with apiserver"
May 25 06:46:02.277599 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.277556    2572 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 25 06:46:02.277599 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.277562    2572 kubelet.go:2451] "Starting kubelet main sync loop"
May 25 06:46:02.277731 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.277607    2572 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 25 06:46:02.280276 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.280260    2572 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 06:46:02.345575 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.345506    2572 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 06:46:02.347706 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.347687    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasSufficientMemory"
May 25 06:46:02.347808 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.347717    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasNoDiskPressure"
May 25 06:46:02.347808 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.347729    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasSufficientPID"
May 25 06:46:02.347808 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.347755    2572 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-128-6.ec2.internal"
May 25 06:46:02.358450 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.358432    2572 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-128-6.ec2.internal"
May 25 06:46:02.358553 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.358457    2572 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-128-6.ec2.internal\": node \"ip-10-0-128-6.ec2.internal\" not found"
May 25 06:46:02.373040 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.373019    2572 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-6.ec2.internal\" not found"
May 25 06:46:02.378307 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.378281    2572 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-128-6.ec2.internal"]
May 25 06:46:02.378404 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.378361    2572 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 06:46:02.379366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.379350    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasSufficientMemory"
May 25 06:46:02.379436 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.379380    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasNoDiskPressure"
May 25 06:46:02.379436 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.379391    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasSufficientPID"
May 25 06:46:02.380530 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.380518    2572 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 06:46:02.380723 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.380690    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.380723 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.380719    2572 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 06:46:02.380859 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.380796    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/630e109509f569b728a57a151f2f6784-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal\" (UID: \"630e109509f569b728a57a151f2f6784\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.380859 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.380813    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/630e109509f569b728a57a151f2f6784-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal\" (UID: \"630e109509f569b728a57a151f2f6784\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.380859 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.380831    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/732aed7a0244464bbcdfac9017fa7a54-config\") pod \"kube-apiserver-proxy-ip-10-0-128-6.ec2.internal\" (UID: \"732aed7a0244464bbcdfac9017fa7a54\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.381298 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.381282    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasSufficientMemory"
May 25 06:46:02.381298 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.381294    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasSufficientMemory"
May 25 06:46:02.381384 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.381309    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasNoDiskPressure"
May 25 06:46:02.381384 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.381314    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasNoDiskPressure"
May 25 06:46:02.381384 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.381321    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasSufficientPID"
May 25 06:46:02.381384 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.381324    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasSufficientPID"
May 25 06:46:02.382470 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.382457    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.382519 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.382482    2572 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 25 06:46:02.383134 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.383120    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasSufficientMemory"
May 25 06:46:02.383183 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.383145    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasNoDiskPressure"
May 25 06:46:02.383183 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.383159    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeHasSufficientPID"
May 25 06:46:02.399408 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.399386    2572 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-6.ec2.internal\" not found" node="ip-10-0-128-6.ec2.internal"
May 25 06:46:02.403698 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.403682    2572 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-128-6.ec2.internal\" not found" node="ip-10-0-128-6.ec2.internal"
May 25 06:46:02.473453 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.473422    2572 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-6.ec2.internal\" not found"
May 25 06:46:02.481284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.481254    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/630e109509f569b728a57a151f2f6784-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal\" (UID: \"630e109509f569b728a57a151f2f6784\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.481425 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.481291    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/630e109509f569b728a57a151f2f6784-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal\" (UID: \"630e109509f569b728a57a151f2f6784\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.481425 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.481352    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/630e109509f569b728a57a151f2f6784-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal\" (UID: \"630e109509f569b728a57a151f2f6784\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.481425 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.481353    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/630e109509f569b728a57a151f2f6784-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal\" (UID: \"630e109509f569b728a57a151f2f6784\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.481425 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.481396    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/732aed7a0244464bbcdfac9017fa7a54-config\") pod \"kube-apiserver-proxy-ip-10-0-128-6.ec2.internal\" (UID: \"732aed7a0244464bbcdfac9017fa7a54\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.481425 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.481419    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/732aed7a0244464bbcdfac9017fa7a54-config\") pod \"kube-apiserver-proxy-ip-10-0-128-6.ec2.internal\" (UID: \"732aed7a0244464bbcdfac9017fa7a54\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.573999 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.573965    2572 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-6.ec2.internal\" not found"
May 25 06:46:02.674749 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.674676    2572 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-6.ec2.internal\" not found"
May 25 06:46:02.702139 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.702109    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.706620 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:02.706591    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-6.ec2.internal"
May 25 06:46:02.775293 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.775257    2572 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-6.ec2.internal\" not found"
May 25 06:46:02.875774 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.875737    2572 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-6.ec2.internal\" not found"
May 25 06:46:02.976311 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:02.976232    2572 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-6.ec2.internal\" not found"
May 25 06:46:03.076798 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.076760    2572 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-128-6.ec2.internal\" not found"
May 25 06:46:03.097250 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.097213    2572 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 25 06:46:03.097444 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.097411    2572 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 25 06:46:03.097444 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.097410    2572 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 25 06:46:03.111422 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.111393    2572 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 06:46:03.160068 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.160039    2572 apiserver.go:52] "Watching apiserver"
May 25 06:46:03.166960 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.166932    2572 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 25 06:46:03.167287 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.167268    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/multus-additional-cni-plugins-5jwq4","openshift-multus/network-metrics-daemon-lqsp4","openshift-network-operator/iptables-alerter-2mdqz","openshift-cluster-node-tuning-operator/tuned-5bxk8","openshift-dns/node-resolver-5gkzs","openshift-image-registry/node-ca-87m8j","openshift-network-diagnostics/network-check-target-pttm7","openshift-ovn-kubernetes/ovnkube-node-nxvdb","kube-system/konnectivity-agent-77nvd","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd","openshift-multus/multus-4tfcl"]
May 25 06:46:03.169799 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.169776    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.171107 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.171084    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:03.171238 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.171207    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:03.173725 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.172981    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.173725 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.173059    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 25 06:46:03.173725 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.173475    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 25 06:46:03.173725 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.173651    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-cxqvz\""
May 25 06:46:03.173725 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.173724    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 25 06:46:03.174107 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.173765    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 25 06:46:03.174107 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.173923    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 25 06:46:03.174991 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.174968    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 25 06:46:03.175120 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.175089    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2mdqz"
May 25 06:46:03.175588 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.175571    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-wmdwf\""
May 25 06:46:03.175924 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.175908    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 25 06:46:03.176247 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.176229    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5gkzs"
May 25 06:46:03.176362 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.176324    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-87m8j"
May 25 06:46:03.176624 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.176608    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-4fgz8\""
May 25 06:46:03.176823 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.176807    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 25 06:46:03.177208 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.177191    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:03.177295 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.177252    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:03.177435 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.177413    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 25 06:46:03.177517 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.177475    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 25 06:46:03.178344 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.178326    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 25 06:46:03.178492 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.178478    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.178748 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.178665    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9mw2g\""
May 25 06:46:03.179052 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.179028    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-ppfj6\""
May 25 06:46:03.179052 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.179039    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 25 06:46:03.179228 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.179159    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 25 06:46:03.179228 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.179168    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 25 06:46:03.179320 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.179247    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 25 06:46:03.179867 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.179834    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-77nvd"
May 25 06:46:03.180460 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.180440    2572 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 25 06:46:03.180962 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.180945    2572 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal"
May 25 06:46:03.181193 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.181177    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 25 06:46:03.181433 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.181419    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 25 06:46:03.181517 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.181504    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.181605 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.181592    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 25 06:46:03.182191 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.182172    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 25 06:46:03.182307 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.182252    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-s4qjm\""
May 25 06:46:03.182376 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.182320    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 25 06:46:03.182476 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.182408    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-bjhdr\""
May 25 06:46:03.182577 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.182557    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 25 06:46:03.182667 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.182554    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-24 06:41:02 +0000 UTC" deadline="2028-01-01 00:45:37.463700905 +0000 UTC"
May 25 06:46:03.182667 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.182607    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="14057h59m34.281111592s"
May 25 06:46:03.182763 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.182683    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.182816 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.182759    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 25 06:46:03.182816 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.182783    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 25 06:46:03.183532 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.183510    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 25 06:46:03.183825 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.183806    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 25 06:46:03.184007 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.183991    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-5g9jm\""
May 25 06:46:03.184300 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.184274    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 25 06:46:03.184370 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.184295    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 25 06:46:03.184425 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.184368    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-dnz6h\""
May 25 06:46:03.185252 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185236    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-run-openvswitch\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.185335 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185262    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cddf423c-dbfd-4f24-95d5-dd6089ee0803-ovn-node-metrics-cert\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.185335 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185279    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-registration-dir\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.185335 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185294    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-systemd-units\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.185335 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185320    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-node-log\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.185513 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185355    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a7c58bab-87c9-44c8-90d8-5983437cd492-konnectivity-ca\") pod \"konnectivity-agent-77nvd\" (UID: \"a7c58bab-87c9-44c8-90d8-5983437cd492\") " pod="kube-system/konnectivity-agent-77nvd"
May 25 06:46:03.185513 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185410    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6aeda4f-547c-488a-9414-e6ab811425c6-system-cni-dir\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.185513 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185443    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6aeda4f-547c-488a-9414-e6ab811425c6-cni-binary-copy\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.185513 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185473    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a6aeda4f-547c-488a-9414-e6ab811425c6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.185703 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185516    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a068851b-398a-41ba-8fec-1afddc1890f5-iptables-alerter-script\") pod \"iptables-alerter-2mdqz\" (UID: \"a068851b-398a-41ba-8fec-1afddc1890f5\") " pod="openshift-network-operator/iptables-alerter-2mdqz"
May 25 06:46:03.185703 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185553    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-lib-modules\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.185703 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185580    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-run-ovn-kubernetes\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.185703 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185604    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xl5bb\" (UniqueName: \"kubernetes.io/projected/cddf423c-dbfd-4f24-95d5-dd6089ee0803-kube-api-access-xl5bb\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.185703 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185640    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.185703 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185670    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-sys-fs\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.185703 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185692    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-var-lib-kubelet\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.185996 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185716    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vvdzf\" (UniqueName: \"kubernetes.io/projected/869a2061-282e-499e-a729-c1df0fc783a3-kube-api-access-vvdzf\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.185996 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185746    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-cni-netd\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.185996 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185770    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cddf423c-dbfd-4f24-95d5-dd6089ee0803-ovnkube-config\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.185996 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185796    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wr8lc\" (UniqueName: \"kubernetes.io/projected/a6aeda4f-547c-488a-9414-e6ab811425c6-kube-api-access-wr8lc\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.185996 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185821    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-kubernetes\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.185996 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185864    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3930217-a45d-4bff-8456-a261f55e2daa-etc-tuned\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.185996 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185893    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9nkw\" (UniqueName: \"kubernetes.io/projected/c3130f84-1146-48b3-97c0-bf676f8eb29d-kube-api-access-p9nkw\") pod \"node-ca-87m8j\" (UID: \"c3130f84-1146-48b3-97c0-bf676f8eb29d\") " pod="openshift-image-registry/node-ca-87m8j"
May 25 06:46:03.185996 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185916    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-run-netns\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.185996 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185941    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a569a74b-5db9-428c-8a4e-9692a5636e5d-tmp-dir\") pod \"node-resolver-5gkzs\" (UID: \"a569a74b-5db9-428c-8a4e-9692a5636e5d\") " pod="openshift-dns/node-resolver-5gkzs"
May 25 06:46:03.185996 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185962    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-run-ovn\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.186392 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.185985    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-cni-bin\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.186392 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186030    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cddf423c-dbfd-4f24-95d5-dd6089ee0803-ovnkube-script-lib\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.186392 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186082    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps5xc\" (UniqueName: \"kubernetes.io/projected/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-kube-api-access-ps5xc\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:03.186392 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186120    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-modprobe-d\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.186392 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186169    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6aeda4f-547c-488a-9414-e6ab811425c6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.186392 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186214    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-socket-dir\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.186392 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186258    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7wv82\" (UniqueName: \"kubernetes.io/projected/d3930217-a45d-4bff-8456-a261f55e2daa-kube-api-access-7wv82\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.186392 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186286    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-kubelet\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.186392 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186314    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-run-systemd\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.186392 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186339    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-etc-openvswitch\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.186392 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186369    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.186392 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186408    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8cbc\" (UniqueName: \"kubernetes.io/projected/a569a74b-5db9-428c-8a4e-9692a5636e5d-kube-api-access-m8cbc\") pod \"node-resolver-5gkzs\" (UID: \"a569a74b-5db9-428c-8a4e-9692a5636e5d\") " pod="openshift-dns/node-resolver-5gkzs"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186447    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186483    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-sysctl-d\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186520    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3130f84-1146-48b3-97c0-bf676f8eb29d-host\") pod \"node-ca-87m8j\" (UID: \"c3130f84-1146-48b3-97c0-bf676f8eb29d\") " pod="openshift-image-registry/node-ca-87m8j"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186542    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-var-lib-openvswitch\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186563    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vkd8f\" (UniqueName: \"kubernetes.io/projected/a068851b-398a-41ba-8fec-1afddc1890f5-kube-api-access-vkd8f\") pod \"iptables-alerter-2mdqz\" (UID: \"a068851b-398a-41ba-8fec-1afddc1890f5\") " pod="openshift-network-operator/iptables-alerter-2mdqz"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186584    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-run\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186605    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3930217-a45d-4bff-8456-a261f55e2daa-tmp\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186639    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3130f84-1146-48b3-97c0-bf676f8eb29d-serviceca\") pod \"node-ca-87m8j\" (UID: \"c3130f84-1146-48b3-97c0-bf676f8eb29d\") " pod="openshift-image-registry/node-ca-87m8j"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186681    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-log-socket\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186731    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cddf423c-dbfd-4f24-95d5-dd6089ee0803-env-overrides\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186776    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a7c58bab-87c9-44c8-90d8-5983437cd492-agent-certs\") pod \"konnectivity-agent-77nvd\" (UID: \"a7c58bab-87c9-44c8-90d8-5983437cd492\") " pod="kube-system/konnectivity-agent-77nvd"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186807    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a6aeda4f-547c-488a-9414-e6ab811425c6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186830    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-slash\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186867    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a569a74b-5db9-428c-8a4e-9692a5636e5d-hosts-file\") pod \"node-resolver-5gkzs\" (UID: \"a569a74b-5db9-428c-8a4e-9692a5636e5d\") " pod="openshift-dns/node-resolver-5gkzs"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186891    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-442st\" (UniqueName: \"kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st\") pod \"network-check-target-pttm7\" (UID: \"50545ca0-b117-4f61-8439-62bd12fbf0c0\") " pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:03.187284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186917    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-sysconfig\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.188028 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186939    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-systemd\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.188028 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186961    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-host\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.188028 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186976    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6aeda4f-547c-488a-9414-e6ab811425c6-cnibin\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.188028 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.186990    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6aeda4f-547c-488a-9414-e6ab811425c6-os-release\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.188028 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.187028    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a068851b-398a-41ba-8fec-1afddc1890f5-host-slash\") pod \"iptables-alerter-2mdqz\" (UID: \"a068851b-398a-41ba-8fec-1afddc1890f5\") " pod="openshift-network-operator/iptables-alerter-2mdqz"
May 25 06:46:03.188028 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.187058    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-sysctl-conf\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.188028 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.187088    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-device-dir\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.188028 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.187108    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:03.188028 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.187122    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-sys\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.192561 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.192540    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 25 06:46:03.192641 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.192631    2572 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-6.ec2.internal"
May 25 06:46:03.192799 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.192784    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal"]
May 25 06:46:03.194219 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.194126    2572 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 25 06:46:03.200050 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.200033    2572 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 25 06:46:03.200205 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.200189    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-128-6.ec2.internal"]
May 25 06:46:03.212003 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:03.211975    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod732aed7a0244464bbcdfac9017fa7a54.slice/crio-1a2b1e6d30bfb31cbb2d0f908e9de4d0f3ac087a06d52c511e91bff515704b60 WatchSource:0}: Error finding container 1a2b1e6d30bfb31cbb2d0f908e9de4d0f3ac087a06d52c511e91bff515704b60: Status 404 returned error can't find the container with id 1a2b1e6d30bfb31cbb2d0f908e9de4d0f3ac087a06d52c511e91bff515704b60
May 25 06:46:03.212219 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:03.212197    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod630e109509f569b728a57a151f2f6784.slice/crio-07d7147164446dc8da27278fc8d437dc265ce988cb9425d218ee72df9307a23d WatchSource:0}: Error finding container 07d7147164446dc8da27278fc8d437dc265ce988cb9425d218ee72df9307a23d: Status 404 returned error can't find the container with id 07d7147164446dc8da27278fc8d437dc265ce988cb9425d218ee72df9307a23d
May 25 06:46:03.217038 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.217017    2572 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-qh8gn"
May 25 06:46:03.218275 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.218250    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 06:46:03.225391 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.225370    2572 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-qh8gn"
May 25 06:46:03.280231 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.280180    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-6.ec2.internal" event={"ID":"732aed7a0244464bbcdfac9017fa7a54","Type":"ContainerStarted","Data":"1a2b1e6d30bfb31cbb2d0f908e9de4d0f3ac087a06d52c511e91bff515704b60"}
May 25 06:46:03.281096 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.281071    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal" event={"ID":"630e109509f569b728a57a151f2f6784","Type":"ContainerStarted","Data":"07d7147164446dc8da27278fc8d437dc265ce988cb9425d218ee72df9307a23d"}
May 25 06:46:03.281243 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.281229    2572 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288446    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288526    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-sys-fs\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288560    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-var-lib-kubelet\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288593    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vvdzf\" (UniqueName: \"kubernetes.io/projected/869a2061-282e-499e-a729-c1df0fc783a3-kube-api-access-vvdzf\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288624    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-multus-socket-dir-parent\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288659    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-hostroot\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288693    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-cni-netd\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288726    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cddf423c-dbfd-4f24-95d5-dd6089ee0803-ovnkube-config\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288763    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wr8lc\" (UniqueName: \"kubernetes.io/projected/a6aeda4f-547c-488a-9414-e6ab811425c6-kube-api-access-wr8lc\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288817    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-run-k8s-cni-cncf-io\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288863    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-run-netns\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288895    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-var-lib-cni-bin\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288926    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-multus-daemon-config\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288960    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-kubernetes\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.288993    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3930217-a45d-4bff-8456-a261f55e2daa-etc-tuned\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289020    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p9nkw\" (UniqueName: \"kubernetes.io/projected/c3130f84-1146-48b3-97c0-bf676f8eb29d-kube-api-access-p9nkw\") pod \"node-ca-87m8j\" (UID: \"c3130f84-1146-48b3-97c0-bf676f8eb29d\") " pod="openshift-image-registry/node-ca-87m8j"
May 25 06:46:03.290303 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289053    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-run-netns\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289088    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a569a74b-5db9-428c-8a4e-9692a5636e5d-tmp-dir\") pod \"node-resolver-5gkzs\" (UID: \"a569a74b-5db9-428c-8a4e-9692a5636e5d\") " pod="openshift-dns/node-resolver-5gkzs"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289120    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-run-ovn\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289152    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-cni-bin\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289184    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cddf423c-dbfd-4f24-95d5-dd6089ee0803-ovnkube-script-lib\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289214    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ps5xc\" (UniqueName: \"kubernetes.io/projected/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-kube-api-access-ps5xc\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289248    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-modprobe-d\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289278    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6aeda4f-547c-488a-9414-e6ab811425c6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289310    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-socket-dir\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289345    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-cni-binary-copy\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289373    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7wv82\" (UniqueName: \"kubernetes.io/projected/d3930217-a45d-4bff-8456-a261f55e2daa-kube-api-access-7wv82\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289407    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-kubelet\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289437    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-run-systemd\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289481    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-etc-openvswitch\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289513    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289546    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m8cbc\" (UniqueName: \"kubernetes.io/projected/a569a74b-5db9-428c-8a4e-9692a5636e5d-kube-api-access-m8cbc\") pod \"node-resolver-5gkzs\" (UID: \"a569a74b-5db9-428c-8a4e-9692a5636e5d\") " pod="openshift-dns/node-resolver-5gkzs"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289580    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.291177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289613    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-sysctl-d\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289647    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3130f84-1146-48b3-97c0-bf676f8eb29d-host\") pod \"node-ca-87m8j\" (UID: \"c3130f84-1146-48b3-97c0-bf676f8eb29d\") " pod="openshift-image-registry/node-ca-87m8j"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289679    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-var-lib-openvswitch\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289711    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vkd8f\" (UniqueName: \"kubernetes.io/projected/a068851b-398a-41ba-8fec-1afddc1890f5-kube-api-access-vkd8f\") pod \"iptables-alerter-2mdqz\" (UID: \"a068851b-398a-41ba-8fec-1afddc1890f5\") " pod="openshift-network-operator/iptables-alerter-2mdqz"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289741    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-run\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289770    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3930217-a45d-4bff-8456-a261f55e2daa-tmp\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289795    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3130f84-1146-48b3-97c0-bf676f8eb29d-serviceca\") pod \"node-ca-87m8j\" (UID: \"c3130f84-1146-48b3-97c0-bf676f8eb29d\") " pod="openshift-image-registry/node-ca-87m8j"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289827    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-log-socket\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289874    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cddf423c-dbfd-4f24-95d5-dd6089ee0803-env-overrides\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289912    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a7c58bab-87c9-44c8-90d8-5983437cd492-agent-certs\") pod \"konnectivity-agent-77nvd\" (UID: \"a7c58bab-87c9-44c8-90d8-5983437cd492\") " pod="kube-system/konnectivity-agent-77nvd"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289944    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a6aeda4f-547c-488a-9414-e6ab811425c6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.289970    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-slash\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290008    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a569a74b-5db9-428c-8a4e-9692a5636e5d-hosts-file\") pod \"node-resolver-5gkzs\" (UID: \"a569a74b-5db9-428c-8a4e-9692a5636e5d\") " pod="openshift-dns/node-resolver-5gkzs"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290040    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-442st\" (UniqueName: \"kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st\") pod \"network-check-target-pttm7\" (UID: \"50545ca0-b117-4f61-8439-62bd12fbf0c0\") " pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290087    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-cnibin\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290118    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-var-lib-cni-multus\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290145    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-var-lib-kubelet\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.291939 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290178    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-sysconfig\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290213    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-etc-kubernetes\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290265    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c3130f84-1146-48b3-97c0-bf676f8eb29d-host\") pod \"node-ca-87m8j\" (UID: \"c3130f84-1146-48b3-97c0-bf676f8eb29d\") " pod="openshift-image-registry/node-ca-87m8j"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290343    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290414    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-systemd\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290454    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-host\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290438    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-run-systemd\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290493    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6aeda4f-547c-488a-9414-e6ab811425c6-cnibin\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290527    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6aeda4f-547c-488a-9414-e6ab811425c6-os-release\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290559    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a068851b-398a-41ba-8fec-1afddc1890f5-host-slash\") pod \"iptables-alerter-2mdqz\" (UID: \"a068851b-398a-41ba-8fec-1afddc1890f5\") " pod="openshift-network-operator/iptables-alerter-2mdqz"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290614    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/a068851b-398a-41ba-8fec-1afddc1890f5-host-slash\") pod \"iptables-alerter-2mdqz\" (UID: \"a068851b-398a-41ba-8fec-1afddc1890f5\") " pod="openshift-network-operator/iptables-alerter-2mdqz"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290639    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-sysctl-d\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290656    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-run-ovn\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290704    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-var-lib-kubelet\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290705    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-kubernetes\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290702    2572 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290748    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-var-lib-openvswitch\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290907    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/a6aeda4f-547c-488a-9414-e6ab811425c6-cnibin\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.292738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290903    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-systemd\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290992    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-run-netns\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.290873    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-host\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291331    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-cni-bin\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291299    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-slash\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291296    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-kubelet\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291334    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/a6aeda4f-547c-488a-9414-e6ab811425c6-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291386    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-sysctl-conf\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291420    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-device-dir\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291438    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-modprobe-d\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291451    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-system-cni-dir\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291525    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/a6aeda4f-547c-488a-9414-e6ab811425c6-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291544    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-cni-netd\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291563    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-socket-dir\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291616    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-device-dir\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291653    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-etc-openvswitch\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291662    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-etc-selinux\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.293563 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291694    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-kubelet-dir\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291701    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-multus-conf-dir\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291734    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9hz2v\" (UniqueName: \"kubernetes.io/projected/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-kube-api-access-9hz2v\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291765    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291782    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/c3130f84-1146-48b3-97c0-bf676f8eb29d-serviceca\") pod \"node-ca-87m8j\" (UID: \"c3130f84-1146-48b3-97c0-bf676f8eb29d\") " pod="openshift-image-registry/node-ca-87m8j"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291793    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-sys\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291820    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-run-openvswitch\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291867    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cddf423c-dbfd-4f24-95d5-dd6089ee0803-ovn-node-metrics-cert\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291946    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-registration-dir\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291978    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-multus-cni-dir\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292008    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-systemd-units\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292035    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-node-log\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292066    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a7c58bab-87c9-44c8-90d8-5983437cd492-konnectivity-ca\") pod \"konnectivity-agent-77nvd\" (UID: \"a7c58bab-87c9-44c8-90d8-5983437cd492\") " pod="kube-system/konnectivity-agent-77nvd"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292095    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6aeda4f-547c-488a-9414-e6ab811425c6-system-cni-dir\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292122    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6aeda4f-547c-488a-9414-e6ab811425c6-cni-binary-copy\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292151    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a6aeda4f-547c-488a-9414-e6ab811425c6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292171    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/cddf423c-dbfd-4f24-95d5-dd6089ee0803-ovnkube-script-lib\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.294264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292199    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-os-release\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292227    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-run-openvswitch\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.291979    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/a6aeda4f-547c-488a-9414-e6ab811425c6-os-release\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292278    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-run-multus-certs\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292290    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/cddf423c-dbfd-4f24-95d5-dd6089ee0803-env-overrides\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292314    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a068851b-398a-41ba-8fec-1afddc1890f5-iptables-alerter-script\") pod \"iptables-alerter-2mdqz\" (UID: \"a068851b-398a-41ba-8fec-1afddc1890f5\") " pod="openshift-network-operator/iptables-alerter-2mdqz"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292346    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-lib-modules\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292376    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-run-ovn-kubernetes\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292405    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xl5bb\" (UniqueName: \"kubernetes.io/projected/cddf423c-dbfd-4f24-95d5-dd6089ee0803-kube-api-access-xl5bb\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292418    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-run\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292492    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a569a74b-5db9-428c-8a4e-9692a5636e5d-tmp-dir\") pod \"node-resolver-5gkzs\" (UID: \"a569a74b-5db9-428c-8a4e-9692a5636e5d\") " pod="openshift-dns/node-resolver-5gkzs"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.292503    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292531    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-sysctl-conf\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292580    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/a569a74b-5db9-428c-8a4e-9692a5636e5d-hosts-file\") pod \"node-resolver-5gkzs\" (UID: \"a569a74b-5db9-428c-8a4e-9692a5636e5d\") " pod="openshift-dns/node-resolver-5gkzs"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292636    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/cddf423c-dbfd-4f24-95d5-dd6089ee0803-ovnkube-config\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292692    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-etc-sysconfig\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.292709    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs podName:28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:03.792680494 +0000 UTC m=+2.028866887 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs") pod "network-metrics-daemon-lqsp4" (UID: "28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:03.294739 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292638    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-systemd-units\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292861    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-sys-fs\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292883    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/869a2061-282e-499e-a729-c1df0fc783a3-registration-dir\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292923    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-node-log\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292958    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/a6aeda4f-547c-488a-9414-e6ab811425c6-system-cni-dir\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.293060    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/a6aeda4f-547c-488a-9414-e6ab811425c6-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.293067    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-lib-modules\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292347    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/d3930217-a45d-4bff-8456-a261f55e2daa-sys\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.292377    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-log-socket\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.293286    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/a7c58bab-87c9-44c8-90d8-5983437cd492-konnectivity-ca\") pod \"konnectivity-agent-77nvd\" (UID: \"a7c58bab-87c9-44c8-90d8-5983437cd492\") " pod="kube-system/konnectivity-agent-77nvd"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.293355    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/cddf423c-dbfd-4f24-95d5-dd6089ee0803-host-run-ovn-kubernetes\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.293506    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/a6aeda4f-547c-488a-9414-e6ab811425c6-cni-binary-copy\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.293578    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/a068851b-398a-41ba-8fec-1afddc1890f5-iptables-alerter-script\") pod \"iptables-alerter-2mdqz\" (UID: \"a068851b-398a-41ba-8fec-1afddc1890f5\") " pod="openshift-network-operator/iptables-alerter-2mdqz"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.295169    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/d3930217-a45d-4bff-8456-a261f55e2daa-tmp\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.295270    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/d3930217-a45d-4bff-8456-a261f55e2daa-etc-tuned\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.295366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.295308    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/cddf423c-dbfd-4f24-95d5-dd6089ee0803-ovn-node-metrics-cert\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.295770 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.295439    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/a7c58bab-87c9-44c8-90d8-5983437cd492-agent-certs\") pod \"konnectivity-agent-77nvd\" (UID: \"a7c58bab-87c9-44c8-90d8-5983437cd492\") " pod="kube-system/konnectivity-agent-77nvd"
May 25 06:46:03.298631 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.298608    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vkd8f\" (UniqueName: \"kubernetes.io/projected/a068851b-398a-41ba-8fec-1afddc1890f5-kube-api-access-vkd8f\") pod \"iptables-alerter-2mdqz\" (UID: \"a068851b-398a-41ba-8fec-1afddc1890f5\") " pod="openshift-network-operator/iptables-alerter-2mdqz"
May 25 06:46:03.299588 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.298998    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 06:46:03.299588 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.299025    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 06:46:03.299588 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.299040    2572 projected.go:194] Error preparing data for projected volume kube-api-access-442st for pod openshift-network-diagnostics/network-check-target-pttm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:03.299588 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.299117    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st podName:50545ca0-b117-4f61-8439-62bd12fbf0c0 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:03.799098434 +0000 UTC m=+2.035284804 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-442st" (UniqueName: "kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st") pod "network-check-target-pttm7" (UID: "50545ca0-b117-4f61-8439-62bd12fbf0c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:03.299588 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.299502    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps5xc\" (UniqueName: \"kubernetes.io/projected/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-kube-api-access-ps5xc\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:03.299990 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.299959    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8cbc\" (UniqueName: \"kubernetes.io/projected/a569a74b-5db9-428c-8a4e-9692a5636e5d-kube-api-access-m8cbc\") pod \"node-resolver-5gkzs\" (UID: \"a569a74b-5db9-428c-8a4e-9692a5636e5d\") " pod="openshift-dns/node-resolver-5gkzs"
May 25 06:46:03.300062 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.300050    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9nkw\" (UniqueName: \"kubernetes.io/projected/c3130f84-1146-48b3-97c0-bf676f8eb29d-kube-api-access-p9nkw\") pod \"node-ca-87m8j\" (UID: \"c3130f84-1146-48b3-97c0-bf676f8eb29d\") " pod="openshift-image-registry/node-ca-87m8j"
May 25 06:46:03.300324 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.300305    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vvdzf\" (UniqueName: \"kubernetes.io/projected/869a2061-282e-499e-a729-c1df0fc783a3-kube-api-access-vvdzf\") pod \"aws-ebs-csi-driver-node-5bqmd\" (UID: \"869a2061-282e-499e-a729-c1df0fc783a3\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.301458 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.301437    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7wv82\" (UniqueName: \"kubernetes.io/projected/d3930217-a45d-4bff-8456-a261f55e2daa-kube-api-access-7wv82\") pod \"tuned-5bxk8\" (UID: \"d3930217-a45d-4bff-8456-a261f55e2daa\") " pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.301571 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.301551    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xl5bb\" (UniqueName: \"kubernetes.io/projected/cddf423c-dbfd-4f24-95d5-dd6089ee0803-kube-api-access-xl5bb\") pod \"ovnkube-node-nxvdb\" (UID: \"cddf423c-dbfd-4f24-95d5-dd6089ee0803\") " pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.301648 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.301628    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wr8lc\" (UniqueName: \"kubernetes.io/projected/a6aeda4f-547c-488a-9414-e6ab811425c6-kube-api-access-wr8lc\") pod \"multus-additional-cni-plugins-5jwq4\" (UID: \"a6aeda4f-547c-488a-9414-e6ab811425c6\") " pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.304919 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.304903    2572 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 06:46:03.393738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393705    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-run-netns\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.393738 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393738    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-var-lib-cni-bin\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393760    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-multus-daemon-config\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393792    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-cni-binary-copy\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393811    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-run-netns\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393853    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-var-lib-cni-bin\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393870    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-cnibin\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393904    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-var-lib-cni-multus\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393930    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-var-lib-kubelet\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393952    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-etc-kubernetes\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393972    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-cnibin\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393987    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-var-lib-cni-multus\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.393979    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-system-cni-dir\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394021    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-etc-kubernetes\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394012    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-var-lib-kubelet\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394022    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-system-cni-dir\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394053 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394037    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-multus-conf-dir\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394069    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9hz2v\" (UniqueName: \"kubernetes.io/projected/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-kube-api-access-9hz2v\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394091    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-multus-conf-dir\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394111    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-multus-cni-dir\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394236    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-os-release\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394262    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-multus-cni-dir\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394266    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-run-multus-certs\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394308    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-os-release\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394314    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-multus-socket-dir-parent\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394341    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-hostroot\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394362    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-multus-socket-dir-parent\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394369    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-run-k8s-cni-cncf-io\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394380    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-run-multus-certs\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394415    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-cni-binary-copy\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394439    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-multus-daemon-config\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394456    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-host-run-k8s-cni-cncf-io\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.394627 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.394488    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-hostroot\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.401501 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.401477    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9hz2v\" (UniqueName: \"kubernetes.io/projected/c2acf7a4-48ea-4c5e-8d4c-d8232f383295-kube-api-access-9hz2v\") pod \"multus-4tfcl\" (UID: \"c2acf7a4-48ea-4c5e-8d4c-d8232f383295\") " pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.499575 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.499494    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5jwq4"
May 25 06:46:03.506244 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:03.506209    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda6aeda4f_547c_488a_9414_e6ab811425c6.slice/crio-179708aefe85e6c63abbfef41b86dd72ff5cdd68bd7e927aa8d5a519141e8f52 WatchSource:0}: Error finding container 179708aefe85e6c63abbfef41b86dd72ff5cdd68bd7e927aa8d5a519141e8f52: Status 404 returned error can't find the container with id 179708aefe85e6c63abbfef41b86dd72ff5cdd68bd7e927aa8d5a519141e8f52
May 25 06:46:03.513144 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.513124    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-5bxk8"
May 25 06:46:03.518907 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:03.518883    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3930217_a45d_4bff_8456_a261f55e2daa.slice/crio-546facd8c36b36a78b1ce0a2f236932c98f7191d88d76e5464c3263b78ba778f WatchSource:0}: Error finding container 546facd8c36b36a78b1ce0a2f236932c98f7191d88d76e5464c3263b78ba778f: Status 404 returned error can't find the container with id 546facd8c36b36a78b1ce0a2f236932c98f7191d88d76e5464c3263b78ba778f
May 25 06:46:03.521798 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.521779    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-2mdqz"
May 25 06:46:03.527292 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.527272    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-5gkzs"
May 25 06:46:03.527948 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:03.527931    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda068851b_398a_41ba_8fec_1afddc1890f5.slice/crio-a03eacc5c5a01f1299c34815617e3b1ea918f8c92d9ff4e3f4c0630c32e95c45 WatchSource:0}: Error finding container a03eacc5c5a01f1299c34815617e3b1ea918f8c92d9ff4e3f4c0630c32e95c45: Status 404 returned error can't find the container with id a03eacc5c5a01f1299c34815617e3b1ea918f8c92d9ff4e3f4c0630c32e95c45
May 25 06:46:03.532781 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.532760    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-87m8j"
May 25 06:46:03.533648 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:03.533624    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda569a74b_5db9_428c_8a4e_9692a5636e5d.slice/crio-41438f354faabbd22ebcb44735e84a3dee78d12f981d0449804ad97cb3c37446 WatchSource:0}: Error finding container 41438f354faabbd22ebcb44735e84a3dee78d12f981d0449804ad97cb3c37446: Status 404 returned error can't find the container with id 41438f354faabbd22ebcb44735e84a3dee78d12f981d0449804ad97cb3c37446
May 25 06:46:03.538805 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.538781    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:03.538988 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:03.538965    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3130f84_1146_48b3_97c0_bf676f8eb29d.slice/crio-8fd11d9995d02563826a0bca6324bd26b470892e0209fe20f5368d4b1abb5bd5 WatchSource:0}: Error finding container 8fd11d9995d02563826a0bca6324bd26b470892e0209fe20f5368d4b1abb5bd5: Status 404 returned error can't find the container with id 8fd11d9995d02563826a0bca6324bd26b470892e0209fe20f5368d4b1abb5bd5
May 25 06:46:03.543879 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.543862    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-77nvd"
May 25 06:46:03.546852 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:03.546820    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcddf423c_dbfd_4f24_95d5_dd6089ee0803.slice/crio-d058a835afbea4d8e42d590678b697108617e741bd924158e25acaa50a53b1c9 WatchSource:0}: Error finding container d058a835afbea4d8e42d590678b697108617e741bd924158e25acaa50a53b1c9: Status 404 returned error can't find the container with id d058a835afbea4d8e42d590678b697108617e741bd924158e25acaa50a53b1c9
May 25 06:46:03.549658 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.549643    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd"
May 25 06:46:03.552645 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:03.552612    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda7c58bab_87c9_44c8_90d8_5983437cd492.slice/crio-bc093bdf7ec67cda68c26481bee15cfc1e817b8e49bcf331e89759d7ca51552b WatchSource:0}: Error finding container bc093bdf7ec67cda68c26481bee15cfc1e817b8e49bcf331e89759d7ca51552b: Status 404 returned error can't find the container with id bc093bdf7ec67cda68c26481bee15cfc1e817b8e49bcf331e89759d7ca51552b
May 25 06:46:03.553384 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.553371    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-4tfcl"
May 25 06:46:03.557175 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:03.557141    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod869a2061_282e_499e_a729_c1df0fc783a3.slice/crio-1a239a300075be504ec45e2f01ffc763a3a7081e892f93c15c8443726c71490a WatchSource:0}: Error finding container 1a239a300075be504ec45e2f01ffc763a3a7081e892f93c15c8443726c71490a: Status 404 returned error can't find the container with id 1a239a300075be504ec45e2f01ffc763a3a7081e892f93c15c8443726c71490a
May 25 06:46:03.559889 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:46:03.559873    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc2acf7a4_48ea_4c5e_8d4c_d8232f383295.slice/crio-be9f94f340082c225325622960295e418a14a811d48bb8400e895950d9f4ac0d WatchSource:0}: Error finding container be9f94f340082c225325622960295e418a14a811d48bb8400e895950d9f4ac0d: Status 404 returned error can't find the container with id be9f94f340082c225325622960295e418a14a811d48bb8400e895950d9f4ac0d
May 25 06:46:03.798531 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.798426    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:03.798689 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.798602    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:03.798689 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.798668    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs podName:28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:04.798648599 +0000 UTC m=+3.034834981 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs") pod "network-metrics-daemon-lqsp4" (UID: "28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:03.899439 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:03.899406    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-442st\" (UniqueName: \"kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st\") pod \"network-check-target-pttm7\" (UID: \"50545ca0-b117-4f61-8439-62bd12fbf0c0\") " pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:03.899611 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.899568    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 06:46:03.899611 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.899588    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 06:46:03.899611 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.899600    2572 projected.go:194] Error preparing data for projected volume kube-api-access-442st for pod openshift-network-diagnostics/network-check-target-pttm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:03.899781 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:03.899668    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st podName:50545ca0-b117-4f61-8439-62bd12fbf0c0 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:04.899648531 +0000 UTC m=+3.135834917 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-442st" (UniqueName: "kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st") pod "network-check-target-pttm7" (UID: "50545ca0-b117-4f61-8439-62bd12fbf0c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:04.049398 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.049146    2572 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 06:46:04.226338 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.226304    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-24 06:41:03 +0000 UTC" deadline="2027-12-01 02:11:36.257423894 +0000 UTC"
May 25 06:46:04.226338 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.226335    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13315h25m32.031093293s"
May 25 06:46:04.300665 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.300579    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd" event={"ID":"869a2061-282e-499e-a729-c1df0fc783a3","Type":"ContainerStarted","Data":"1a239a300075be504ec45e2f01ffc763a3a7081e892f93c15c8443726c71490a"}
May 25 06:46:04.304984 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.303605    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-77nvd" event={"ID":"a7c58bab-87c9-44c8-90d8-5983437cd492","Type":"ContainerStarted","Data":"bc093bdf7ec67cda68c26481bee15cfc1e817b8e49bcf331e89759d7ca51552b"}
May 25 06:46:04.324762 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.324697    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5gkzs" event={"ID":"a569a74b-5db9-428c-8a4e-9692a5636e5d","Type":"ContainerStarted","Data":"41438f354faabbd22ebcb44735e84a3dee78d12f981d0449804ad97cb3c37446"}
May 25 06:46:04.336241 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.336173    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2mdqz" event={"ID":"a068851b-398a-41ba-8fec-1afddc1890f5","Type":"ContainerStarted","Data":"a03eacc5c5a01f1299c34815617e3b1ea918f8c92d9ff4e3f4c0630c32e95c45"}
May 25 06:46:04.354243 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.354120    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jwq4" event={"ID":"a6aeda4f-547c-488a-9414-e6ab811425c6","Type":"ContainerStarted","Data":"179708aefe85e6c63abbfef41b86dd72ff5cdd68bd7e927aa8d5a519141e8f52"}
May 25 06:46:04.380040 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.379990    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4tfcl" event={"ID":"c2acf7a4-48ea-4c5e-8d4c-d8232f383295","Type":"ContainerStarted","Data":"be9f94f340082c225325622960295e418a14a811d48bb8400e895950d9f4ac0d"}
May 25 06:46:04.404638 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.404483    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb" event={"ID":"cddf423c-dbfd-4f24-95d5-dd6089ee0803","Type":"ContainerStarted","Data":"d058a835afbea4d8e42d590678b697108617e741bd924158e25acaa50a53b1c9"}
May 25 06:46:04.412065 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.412009    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-87m8j" event={"ID":"c3130f84-1146-48b3-97c0-bf676f8eb29d","Type":"ContainerStarted","Data":"8fd11d9995d02563826a0bca6324bd26b470892e0209fe20f5368d4b1abb5bd5"}
May 25 06:46:04.451381 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.451320    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5bxk8" event={"ID":"d3930217-a45d-4bff-8456-a261f55e2daa","Type":"ContainerStarted","Data":"546facd8c36b36a78b1ce0a2f236932c98f7191d88d76e5464c3263b78ba778f"}
May 25 06:46:04.691542 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.691463    2572 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 25 06:46:04.807290 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.807255    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:04.807463 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:04.807449    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:04.807539 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:04.807512    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs podName:28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:06.807493243 +0000 UTC m=+5.043679620 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs") pod "network-metrics-daemon-lqsp4" (UID: "28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:04.907749 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:04.907715    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-442st\" (UniqueName: \"kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st\") pod \"network-check-target-pttm7\" (UID: \"50545ca0-b117-4f61-8439-62bd12fbf0c0\") " pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:04.908069 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:04.907885    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 06:46:04.908069 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:04.907904    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 06:46:04.908069 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:04.907916    2572 projected.go:194] Error preparing data for projected volume kube-api-access-442st for pod openshift-network-diagnostics/network-check-target-pttm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:04.908069 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:04.907976    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st podName:50545ca0-b117-4f61-8439-62bd12fbf0c0 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:06.907955112 +0000 UTC m=+5.144141485 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-442st" (UniqueName: "kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st") pod "network-check-target-pttm7" (UID: "50545ca0-b117-4f61-8439-62bd12fbf0c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:05.229515 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:05.229449    2572 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-24 06:41:03 +0000 UTC" deadline="2027-10-23 10:29:41.665988549 +0000 UTC"
May 25 06:46:05.229515 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:05.229483    2572 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12387h43m36.436509327s"
May 25 06:46:05.279445 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:05.278713    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:05.279445 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:05.278874    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:05.279445 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:05.279296    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:05.279445 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:05.279389    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:06.823541 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:06.823431    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:06.824007 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:06.823578    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:06.824007 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:06.823651    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs podName:28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:10.823630629 +0000 UTC m=+9.059817003 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs") pod "network-metrics-daemon-lqsp4" (UID: "28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:06.923818 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:06.923782    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-442st\" (UniqueName: \"kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st\") pod \"network-check-target-pttm7\" (UID: \"50545ca0-b117-4f61-8439-62bd12fbf0c0\") " pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:06.924009 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:06.923985    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 06:46:06.924090 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:06.924015    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 06:46:06.924090 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:06.924028    2572 projected.go:194] Error preparing data for projected volume kube-api-access-442st for pod openshift-network-diagnostics/network-check-target-pttm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:06.924189 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:06.924097    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st podName:50545ca0-b117-4f61-8439-62bd12fbf0c0 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:10.924076434 +0000 UTC m=+9.160262816 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-442st" (UniqueName: "kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st") pod "network-check-target-pttm7" (UID: "50545ca0-b117-4f61-8439-62bd12fbf0c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:07.278494 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:07.278379    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:07.278494 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:07.278441    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:07.278782 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:07.278567    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:07.278782 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:07.278693    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:09.278220 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:09.278183    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:09.278690 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:09.278324    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:09.278690 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:09.278631    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:09.278801 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:09.278773    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:10.857755 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:10.857713    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:10.858229 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:10.857912    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:10.858229 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:10.858003    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs podName:28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:18.857981424 +0000 UTC m=+17.094167803 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs") pod "network-metrics-daemon-lqsp4" (UID: "28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:10.958242 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:10.958199    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-442st\" (UniqueName: \"kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st\") pod \"network-check-target-pttm7\" (UID: \"50545ca0-b117-4f61-8439-62bd12fbf0c0\") " pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:10.958443 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:10.958375    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 06:46:10.958443 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:10.958396    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 06:46:10.958443 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:10.958408    2572 projected.go:194] Error preparing data for projected volume kube-api-access-442st for pod openshift-network-diagnostics/network-check-target-pttm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:10.958596 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:10.958489    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st podName:50545ca0-b117-4f61-8439-62bd12fbf0c0 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:18.958466569 +0000 UTC m=+17.194652987 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-442st" (UniqueName: "kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st") pod "network-check-target-pttm7" (UID: "50545ca0-b117-4f61-8439-62bd12fbf0c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:11.278227 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:11.278035    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:11.278227 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:11.278035    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:11.278227 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:11.278223    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:11.278511 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:11.278340    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:13.278127 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:13.278082    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:13.278534 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:13.278148    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:13.278534 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:13.278273    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:13.278534 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:13.278394    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:15.278297 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:15.278261    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:15.278763 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:15.278261    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:15.278763 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:15.278384    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:15.278763 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:15.278489    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:17.278462 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:17.278428    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:17.278947 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:17.278563    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:17.278947 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:17.278652    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:17.278947 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:17.278806    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:18.914855 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:18.914815    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:18.915356 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:18.914958    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:18.915356 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:18.915035    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs podName:28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:34.915014105 +0000 UTC m=+33.151200482 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs") pod "network-metrics-daemon-lqsp4" (UID: "28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:19.015373 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:19.015335    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-442st\" (UniqueName: \"kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st\") pod \"network-check-target-pttm7\" (UID: \"50545ca0-b117-4f61-8439-62bd12fbf0c0\") " pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:19.015564 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:19.015514    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 06:46:19.015564 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:19.015537    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 06:46:19.015564 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:19.015552    2572 projected.go:194] Error preparing data for projected volume kube-api-access-442st for pod openshift-network-diagnostics/network-check-target-pttm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:19.015720 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:19.015615    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st podName:50545ca0-b117-4f61-8439-62bd12fbf0c0 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:35.015600046 +0000 UTC m=+33.251786422 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-442st" (UniqueName: "kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st") pod "network-check-target-pttm7" (UID: "50545ca0-b117-4f61-8439-62bd12fbf0c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:19.278751 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:19.278668    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:19.278924 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:19.278667    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:19.278924 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:19.278790    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:19.279016 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:19.278916    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:21.278708 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:21.278488    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:21.279165 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:21.278493    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:21.279165 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:21.278791    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:21.279165 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:21.278905    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:23.278692 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.278519    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:23.279376 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.278549    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:23.279376 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:23.278765    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:23.279376 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:23.278898    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:23.505697 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.505664    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-6.ec2.internal" event={"ID":"732aed7a0244464bbcdfac9017fa7a54","Type":"ContainerStarted","Data":"31ced9a4ef2c616f90d353fb0e3a537efa3c2ac139e520144865ad2a9967d217"}
May 25 06:46:23.506941 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.506916    2572 generic.go:358] "Generic (PLEG): container finished" podID="630e109509f569b728a57a151f2f6784" containerID="25fb0c51c1a6ad3776b1c260e84da2214aa921a7fd08c4305bce68df10cf0952" exitCode=0
May 25 06:46:23.507038 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.506976    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal" event={"ID":"630e109509f569b728a57a151f2f6784","Type":"ContainerDied","Data":"25fb0c51c1a6ad3776b1c260e84da2214aa921a7fd08c4305bce68df10cf0952"}
May 25 06:46:23.508156 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.508134    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd" event={"ID":"869a2061-282e-499e-a729-c1df0fc783a3","Type":"ContainerStarted","Data":"a6eea1224636455d0bf4e5711bf693774d9a4a3b4a0feef9e0644676800b1a8a"}
May 25 06:46:23.509259 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.509240    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-77nvd" event={"ID":"a7c58bab-87c9-44c8-90d8-5983437cd492","Type":"ContainerStarted","Data":"21a57d1f28ce5f94298fd4bb70607aa50559fc6564fc37f485dd3584aa909c1a"}
May 25 06:46:23.510346 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.510326    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-5gkzs" event={"ID":"a569a74b-5db9-428c-8a4e-9692a5636e5d","Type":"ContainerStarted","Data":"32ab5e15f3fbe7863ea84fcc32bca6f2274d978ac0b7e0429dbdbb9e602770eb"}
May 25 06:46:23.511457 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.511437    2572 generic.go:358] "Generic (PLEG): container finished" podID="a6aeda4f-547c-488a-9414-e6ab811425c6" containerID="e26f9431477b1a0b908a8912af0c0d41be9cf2e01cc3b762baf8c56f71e038e6" exitCode=0
May 25 06:46:23.511537 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.511492    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jwq4" event={"ID":"a6aeda4f-547c-488a-9414-e6ab811425c6","Type":"ContainerDied","Data":"e26f9431477b1a0b908a8912af0c0d41be9cf2e01cc3b762baf8c56f71e038e6"}
May 25 06:46:23.512897 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.512875    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-4tfcl" event={"ID":"c2acf7a4-48ea-4c5e-8d4c-d8232f383295","Type":"ContainerStarted","Data":"09d0bd4b63c38b15ba688c93f9978100d84e297e6dbc6d681508de224a2ff241"}
May 25 06:46:23.515285 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.515270    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 06:46:23.515552 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.515537    2572 generic.go:358] "Generic (PLEG): container finished" podID="cddf423c-dbfd-4f24-95d5-dd6089ee0803" containerID="040ca08f39dde3e1952fda1ee922d455fbc8eab78c436db280229487738f5472" exitCode=1
May 25 06:46:23.515630 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.515585    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb" event={"ID":"cddf423c-dbfd-4f24-95d5-dd6089ee0803","Type":"ContainerStarted","Data":"2f282583b1e375619930a25af4f47b2137b48fb053f623be0d2a5036fb8601e7"}
May 25 06:46:23.515630 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.515603    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb" event={"ID":"cddf423c-dbfd-4f24-95d5-dd6089ee0803","Type":"ContainerStarted","Data":"aedf6434960f6a5e20a29e97f9d7613236165393b12d60b0581cf01657ffac0e"}
May 25 06:46:23.515710 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.515632    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb" event={"ID":"cddf423c-dbfd-4f24-95d5-dd6089ee0803","Type":"ContainerStarted","Data":"18e30bd24952b47db0fe1d3379bca139f948fa4be9689017ce12926a3a9939e6"}
May 25 06:46:23.515710 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.515645    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb" event={"ID":"cddf423c-dbfd-4f24-95d5-dd6089ee0803","Type":"ContainerStarted","Data":"871baeb0ad6aa2f5a291605587d0b91a9afab1057e3ceecb9f7850e46a2b57da"}
May 25 06:46:23.515710 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.515658    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb" event={"ID":"cddf423c-dbfd-4f24-95d5-dd6089ee0803","Type":"ContainerDied","Data":"040ca08f39dde3e1952fda1ee922d455fbc8eab78c436db280229487738f5472"}
May 25 06:46:23.515710 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.515670    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb" event={"ID":"cddf423c-dbfd-4f24-95d5-dd6089ee0803","Type":"ContainerStarted","Data":"97a1b0e50a2c6502786d440f563acbd95df97a65fdfd85829235f53f6d201440"}
May 25 06:46:23.516784 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.516763    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-87m8j" event={"ID":"c3130f84-1146-48b3-97c0-bf676f8eb29d","Type":"ContainerStarted","Data":"8964bfa563bed0a52e836024c5c30ae0543cf0b038a96f48858648ab87cdddd7"}
May 25 06:46:23.518088 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.518067    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-5bxk8" event={"ID":"d3930217-a45d-4bff-8456-a261f55e2daa","Type":"ContainerStarted","Data":"6f867b1bf858fccc05cb892adff2e60098864a91882e0e6f377252734afeea50"}
May 25 06:46:23.519186 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.519149    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-128-6.ec2.internal" podStartSLOduration=20.519139068 podStartE2EDuration="20.519139068s" podCreationTimestamp="2026-05-25 06:46:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 06:46:23.518949237 +0000 UTC m=+21.755135628" watchObservedRunningTime="2026-05-25 06:46:23.519139068 +0000 UTC m=+21.755325460"
May 25 06:46:23.568031 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.567978    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-5gkzs" podStartSLOduration=2.698733991 podStartE2EDuration="21.56796321s" podCreationTimestamp="2026-05-25 06:46:02 +0000 UTC" firstStartedPulling="2026-05-25 06:46:03.535693513 +0000 UTC m=+1.771879883" lastFinishedPulling="2026-05-25 06:46:22.404922731 +0000 UTC m=+20.641109102" observedRunningTime="2026-05-25 06:46:23.55208288 +0000 UTC m=+21.788269274" watchObservedRunningTime="2026-05-25 06:46:23.56796321 +0000 UTC m=+21.804149602"
May 25 06:46:23.568484 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.568451    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-4tfcl" podStartSLOduration=2.256733571 podStartE2EDuration="21.568441106s" podCreationTimestamp="2026-05-25 06:46:02 +0000 UTC" firstStartedPulling="2026-05-25 06:46:03.561310516 +0000 UTC m=+1.797496885" lastFinishedPulling="2026-05-25 06:46:22.873018035 +0000 UTC m=+21.109204420" observedRunningTime="2026-05-25 06:46:23.567633711 +0000 UTC m=+21.803820113" watchObservedRunningTime="2026-05-25 06:46:23.568441106 +0000 UTC m=+21.804627501"
May 25 06:46:23.582263 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.582204    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-5bxk8" podStartSLOduration=2.637089547 podStartE2EDuration="21.582187435s" podCreationTimestamp="2026-05-25 06:46:02 +0000 UTC" firstStartedPulling="2026-05-25 06:46:03.520414416 +0000 UTC m=+1.756600786" lastFinishedPulling="2026-05-25 06:46:22.465512301 +0000 UTC m=+20.701698674" observedRunningTime="2026-05-25 06:46:23.582137707 +0000 UTC m=+21.818324099" watchObservedRunningTime="2026-05-25 06:46:23.582187435 +0000 UTC m=+21.818373827"
May 25 06:46:23.595596 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.595544    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-87m8j" podStartSLOduration=2.6743843529999998 podStartE2EDuration="21.595525885s" podCreationTimestamp="2026-05-25 06:46:02 +0000 UTC" firstStartedPulling="2026-05-25 06:46:03.541216158 +0000 UTC m=+1.777402529" lastFinishedPulling="2026-05-25 06:46:22.462357684 +0000 UTC m=+20.698544061" observedRunningTime="2026-05-25 06:46:23.595424225 +0000 UTC m=+21.831610618" watchObservedRunningTime="2026-05-25 06:46:23.595525885 +0000 UTC m=+21.831712281"
May 25 06:46:23.608389 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:23.608340    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-77nvd" podStartSLOduration=2.7005188650000003 podStartE2EDuration="21.608322647s" podCreationTimestamp="2026-05-25 06:46:02 +0000 UTC" firstStartedPulling="2026-05-25 06:46:03.554029384 +0000 UTC m=+1.790215754" lastFinishedPulling="2026-05-25 06:46:22.461833152 +0000 UTC m=+20.698019536" observedRunningTime="2026-05-25 06:46:23.607723189 +0000 UTC m=+21.843909583" watchObservedRunningTime="2026-05-25 06:46:23.608322647 +0000 UTC m=+21.844509042"
May 25 06:46:24.238030 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:24.237805    2572 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 25 06:46:24.257707 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:24.257565    2572 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-25T06:46:24.238032099Z","UUID":"2c673792-3112-49bd-8fbf-faf731592800","Handler":null,"Name":"","Endpoint":""}
May 25 06:46:24.259295 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:24.259273    2572 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 25 06:46:24.259417 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:24.259302    2572 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 25 06:46:24.523798 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:24.523719    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd" event={"ID":"869a2061-282e-499e-a729-c1df0fc783a3","Type":"ContainerStarted","Data":"07c0ec331d0041c2ecc4f6f9a911432ab13eb901a44c77bfc5004cc3125ba404"}
May 25 06:46:24.525455 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:24.525336    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-2mdqz" event={"ID":"a068851b-398a-41ba-8fec-1afddc1890f5","Type":"ContainerStarted","Data":"08e89dd1c65a8c90837f674dcffdfd824efb10154c572885955e302280b75eba"}
May 25 06:46:24.548398 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:24.548318    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-2mdqz" podStartSLOduration=3.6168198350000003 podStartE2EDuration="22.548299691s" podCreationTimestamp="2026-05-25 06:46:02 +0000 UTC" firstStartedPulling="2026-05-25 06:46:03.530343868 +0000 UTC m=+1.766530237" lastFinishedPulling="2026-05-25 06:46:22.461823712 +0000 UTC m=+20.698010093" observedRunningTime="2026-05-25 06:46:24.547949588 +0000 UTC m=+22.784136004" watchObservedRunningTime="2026-05-25 06:46:24.548299691 +0000 UTC m=+22.784486086"
May 25 06:46:24.700315 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:24.700276    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-77nvd"
May 25 06:46:25.278353 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:25.278320    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:25.278518 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:25.278320    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:25.278518 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:25.278441    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:25.278518 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:25.278486    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:25.530599 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:25.530405    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal" event={"ID":"630e109509f569b728a57a151f2f6784","Type":"ContainerStarted","Data":"9823b73238430cf3caea3d6d3b057007b3ba05bce3d28a78b31b5521ba1d1187"}
May 25 06:46:25.549453 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:25.549408    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-128-6.ec2.internal" podStartSLOduration=22.549394728 podStartE2EDuration="22.549394728s" podCreationTimestamp="2026-05-25 06:46:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 06:46:25.548517162 +0000 UTC m=+23.784703554" watchObservedRunningTime="2026-05-25 06:46:25.549394728 +0000 UTC m=+23.785581119"
May 25 06:46:26.536058 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:26.536027    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 06:46:26.536551 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:26.536396    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb" event={"ID":"cddf423c-dbfd-4f24-95d5-dd6089ee0803","Type":"ContainerStarted","Data":"1b572eddda16d97bf558e7faa6a0e139e307c0087c392ff7623c4dad3ebc9cc1"}
May 25 06:46:26.538416 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:26.538372    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd" event={"ID":"869a2061-282e-499e-a729-c1df0fc783a3","Type":"ContainerStarted","Data":"e79f8e74ac9c129d1310faf76de2cf99b78f70a34cc1f3c103e36240ca20f978"}
May 25 06:46:26.553955 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:26.553898    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-5bqmd" podStartSLOduration=2.681538037 podStartE2EDuration="24.553879139s" podCreationTimestamp="2026-05-25 06:46:02 +0000 UTC" firstStartedPulling="2026-05-25 06:46:03.558719944 +0000 UTC m=+1.794906314" lastFinishedPulling="2026-05-25 06:46:25.43106104 +0000 UTC m=+23.667247416" observedRunningTime="2026-05-25 06:46:26.553571143 +0000 UTC m=+24.789757534" watchObservedRunningTime="2026-05-25 06:46:26.553879139 +0000 UTC m=+24.790065561"
May 25 06:46:27.278203 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:27.278162    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:27.278401 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:27.278184    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:27.278401 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:27.278313    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:27.278401 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:27.278332    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:28.269092 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:28.268869    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-77nvd"
May 25 06:46:28.269647 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:28.269622    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-77nvd"
May 25 06:46:28.543015 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:28.542927    2572 generic.go:358] "Generic (PLEG): container finished" podID="a6aeda4f-547c-488a-9414-e6ab811425c6" containerID="59746c10ec1196759123d815fcc63727d5dcfb312a8547fcdd41b15792a7e871" exitCode=0
May 25 06:46:28.543146 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:28.543010    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jwq4" event={"ID":"a6aeda4f-547c-488a-9414-e6ab811425c6","Type":"ContainerDied","Data":"59746c10ec1196759123d815fcc63727d5dcfb312a8547fcdd41b15792a7e871"}
May 25 06:46:28.546204 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:28.546182    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 06:46:28.546614 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:28.546574    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb" event={"ID":"cddf423c-dbfd-4f24-95d5-dd6089ee0803","Type":"ContainerStarted","Data":"04d0c4cb8eba5a14aad49565fa86b7a3720c9596cb42354522a79e8a3489ecd1"}
May 25 06:46:28.547038 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:28.547017    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:28.547139 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:28.547048    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:28.547263 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:28.547243    2572 scope.go:117] "RemoveContainer" containerID="040ca08f39dde3e1952fda1ee922d455fbc8eab78c436db280229487738f5472"
May 25 06:46:28.547359 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:28.547315    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-77nvd"
May 25 06:46:28.564245 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:28.564227    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:29.278553 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.278529    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:29.278553 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.278563    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:29.279068 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:29.278628    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:29.279068 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:29.278715    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:29.550896 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.550655    2572 generic.go:358] "Generic (PLEG): container finished" podID="a6aeda4f-547c-488a-9414-e6ab811425c6" containerID="1efae4e6f30c851de5b5b7347294ced89a7e57600dcc71e77526cc45b310288e" exitCode=0
May 25 06:46:29.551039 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.550732    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jwq4" event={"ID":"a6aeda4f-547c-488a-9414-e6ab811425c6","Type":"ContainerDied","Data":"1efae4e6f30c851de5b5b7347294ced89a7e57600dcc71e77526cc45b310288e"}
May 25 06:46:29.554355 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.554331    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 06:46:29.554772 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.554746    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb" event={"ID":"cddf423c-dbfd-4f24-95d5-dd6089ee0803","Type":"ContainerStarted","Data":"761e246e27c049da5532127203aefb62127672ea30baa2f3a5437e311f22e2ea"}
May 25 06:46:29.555030 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.554997    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:29.570376 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.570341    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:46:29.591766 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.591712    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb" podStartSLOduration=8.463276206 podStartE2EDuration="27.591694531s" podCreationTimestamp="2026-05-25 06:46:02 +0000 UTC" firstStartedPulling="2026-05-25 06:46:03.548305031 +0000 UTC m=+1.784491401" lastFinishedPulling="2026-05-25 06:46:22.676723342 +0000 UTC m=+20.912909726" observedRunningTime="2026-05-25 06:46:29.591272106 +0000 UTC m=+27.827458497" watchObservedRunningTime="2026-05-25 06:46:29.591694531 +0000 UTC m=+27.827880923"
May 25 06:46:29.973666 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.973630    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pttm7"]
May 25 06:46:29.973943 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.973776    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:29.973943 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:29.973931    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:29.976903 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.976878    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lqsp4"]
May 25 06:46:29.977043 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:29.976983    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:29.977106 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:29.977060    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:30.558171 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:30.558080    2572 generic.go:358] "Generic (PLEG): container finished" podID="a6aeda4f-547c-488a-9414-e6ab811425c6" containerID="1e3073a0974371c2ab9275f98a98be51776b8ed550392c7743ece968fb572e7e" exitCode=0
May 25 06:46:30.558566 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:30.558165    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jwq4" event={"ID":"a6aeda4f-547c-488a-9414-e6ab811425c6","Type":"ContainerDied","Data":"1e3073a0974371c2ab9275f98a98be51776b8ed550392c7743ece968fb572e7e"}
May 25 06:46:31.278284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:31.278243    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:31.278475 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:31.278406    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:32.278973 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:32.278940    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:32.279424 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:32.279043    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:33.277914 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:33.277878    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:33.278109 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:33.278016    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:34.278726 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:34.278681    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:34.279252 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:34.278855    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-pttm7" podUID="50545ca0-b117-4f61-8439-62bd12fbf0c0"
May 25 06:46:34.943644 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:34.943605    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:34.943871 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:34.943776    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:34.943953 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:34.943877    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs podName:28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:06.943854562 +0000 UTC m=+65.180040948 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs") pod "network-metrics-daemon-lqsp4" (UID: "28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 25 06:46:35.044575 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.044534    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-442st\" (UniqueName: \"kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st\") pod \"network-check-target-pttm7\" (UID: \"50545ca0-b117-4f61-8439-62bd12fbf0c0\") " pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:35.044756 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:35.044718    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 25 06:46:35.044756 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:35.044747    2572 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 25 06:46:35.044933 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:35.044759    2572 projected.go:194] Error preparing data for projected volume kube-api-access-442st for pod openshift-network-diagnostics/network-check-target-pttm7: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:35.044933 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:35.044819    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st podName:50545ca0-b117-4f61-8439-62bd12fbf0c0 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:07.044799479 +0000 UTC m=+65.280985858 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-442st" (UniqueName: "kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st") pod "network-check-target-pttm7" (UID: "50545ca0-b117-4f61-8439-62bd12fbf0c0") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 25 06:46:35.278704 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.278615    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:35.278916 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:35.278762    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-lqsp4" podUID="28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3"
May 25 06:46:35.585603 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.585579    2572 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-128-6.ec2.internal" event="NodeReady"
May 25 06:46:35.585887 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.585717    2572 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 25 06:46:35.626228 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.626196    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-gz8v5"]
May 25 06:46:35.681690 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.681611    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-x4v6p"]
May 25 06:46:35.681898 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.681781    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:35.683702 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.683679    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 25 06:46:35.683813 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.683707    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5nl6q\""
May 25 06:46:35.683813 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.683707    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 25 06:46:35.696681 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.696605    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gz8v5"]
May 25 06:46:35.696782 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.696706    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x4v6p"]
May 25 06:46:35.696782 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.696747    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:46:35.698605 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.698586    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 25 06:46:35.698710 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.698611    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 25 06:46:35.698970 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.698931    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-m6t2j\""
May 25 06:46:35.699111 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.699093    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 25 06:46:35.851328 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.851240    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:46:35.851328 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.851301    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-config-volume\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:35.851555 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.851370    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:35.851555 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.851436    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nhdks\" (UniqueName: \"kubernetes.io/projected/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-kube-api-access-nhdks\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:35.851555 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.851469    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-tmp-dir\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:35.851555 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.851551    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lqzdq\" (UniqueName: \"kubernetes.io/projected/f1ba578b-9195-4d91-be43-38f7a5820121-kube-api-access-lqzdq\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:46:35.952623 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.952587    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-config-volume\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:35.952805 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.952631    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:35.952805 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.952660    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nhdks\" (UniqueName: \"kubernetes.io/projected/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-kube-api-access-nhdks\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:35.952805 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:35.952763    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 06:46:35.953010 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:35.952870    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls podName:88e807ae-7d7e-426f-a3ab-25fa0d336a7f nodeName:}" failed. No retries permitted until 2026-05-25 06:46:36.452824128 +0000 UTC m=+34.689010512 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls") pod "dns-default-gz8v5" (UID: "88e807ae-7d7e-426f-a3ab-25fa0d336a7f") : secret "dns-default-metrics-tls" not found
May 25 06:46:35.953010 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.952900    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-tmp-dir\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:35.953010 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.952982    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lqzdq\" (UniqueName: \"kubernetes.io/projected/f1ba578b-9195-4d91-be43-38f7a5820121-kube-api-access-lqzdq\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:46:35.953141 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.953026    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:46:35.953141 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:35.953098    2572 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 06:46:35.953220 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:35.953155    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert podName:f1ba578b-9195-4d91-be43-38f7a5820121 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:36.453138372 +0000 UTC m=+34.689324743 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert") pod "ingress-canary-x4v6p" (UID: "f1ba578b-9195-4d91-be43-38f7a5820121") : secret "canary-serving-cert" not found
May 25 06:46:35.953292 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.953272    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-config-volume\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:35.953334 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.953286    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-tmp-dir\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:35.964279 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.964248    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nhdks\" (UniqueName: \"kubernetes.io/projected/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-kube-api-access-nhdks\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:35.964520 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:35.964499    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lqzdq\" (UniqueName: \"kubernetes.io/projected/f1ba578b-9195-4d91-be43-38f7a5820121-kube-api-access-lqzdq\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:46:36.278454 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:36.278365    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:46:36.280691 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:36.280670    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 25 06:46:36.281060 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:36.280670    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 25 06:46:36.281134 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:36.281117    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nmwpq\""
May 25 06:46:36.456667 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:36.456636    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:36.456967 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:36.456859    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 06:46:36.456967 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:36.456879    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:46:36.456967 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:36.456933    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls podName:88e807ae-7d7e-426f-a3ab-25fa0d336a7f nodeName:}" failed. No retries permitted until 2026-05-25 06:46:37.456914042 +0000 UTC m=+35.693100425 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls") pod "dns-default-gz8v5" (UID: "88e807ae-7d7e-426f-a3ab-25fa0d336a7f") : secret "dns-default-metrics-tls" not found
May 25 06:46:36.457164 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:36.456999    2572 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 06:46:36.457164 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:36.457067    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert podName:f1ba578b-9195-4d91-be43-38f7a5820121 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:37.457049457 +0000 UTC m=+35.693235845 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert") pod "ingress-canary-x4v6p" (UID: "f1ba578b-9195-4d91-be43-38f7a5820121") : secret "canary-serving-cert" not found
May 25 06:46:36.572961 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:36.572927    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jwq4" event={"ID":"a6aeda4f-547c-488a-9414-e6ab811425c6","Type":"ContainerStarted","Data":"7142db8bb72ad04817690bc40552362387346efbb87dd1ec06b0b7d65e28b8a8"}
May 25 06:46:37.278642 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:37.278614    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:46:37.280730 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:37.280704    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 25 06:46:37.281123 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:37.280714    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t49r8\""
May 25 06:46:37.464426 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:37.464389    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:37.464620 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:37.464467    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:46:37.464620 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:37.464541    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 06:46:37.464620 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:37.464564    2572 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 06:46:37.464620 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:37.464608    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls podName:88e807ae-7d7e-426f-a3ab-25fa0d336a7f nodeName:}" failed. No retries permitted until 2026-05-25 06:46:39.464592009 +0000 UTC m=+37.700778383 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls") pod "dns-default-gz8v5" (UID: "88e807ae-7d7e-426f-a3ab-25fa0d336a7f") : secret "dns-default-metrics-tls" not found
May 25 06:46:37.464620 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:37.464624    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert podName:f1ba578b-9195-4d91-be43-38f7a5820121 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:39.464617668 +0000 UTC m=+37.700804038 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert") pod "ingress-canary-x4v6p" (UID: "f1ba578b-9195-4d91-be43-38f7a5820121") : secret "canary-serving-cert" not found
May 25 06:46:37.577523 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:37.577492    2572 generic.go:358] "Generic (PLEG): container finished" podID="a6aeda4f-547c-488a-9414-e6ab811425c6" containerID="7142db8bb72ad04817690bc40552362387346efbb87dd1ec06b0b7d65e28b8a8" exitCode=0
May 25 06:46:37.577676 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:37.577550    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jwq4" event={"ID":"a6aeda4f-547c-488a-9414-e6ab811425c6","Type":"ContainerDied","Data":"7142db8bb72ad04817690bc40552362387346efbb87dd1ec06b0b7d65e28b8a8"}
May 25 06:46:38.581506 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:38.581477    2572 generic.go:358] "Generic (PLEG): container finished" podID="a6aeda4f-547c-488a-9414-e6ab811425c6" containerID="50306e4f57da5bded1f3a43bbe9f6a41e4411ca1519f4c0c8095b15fbc55b887" exitCode=0
May 25 06:46:38.582092 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:38.581512    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jwq4" event={"ID":"a6aeda4f-547c-488a-9414-e6ab811425c6","Type":"ContainerDied","Data":"50306e4f57da5bded1f3a43bbe9f6a41e4411ca1519f4c0c8095b15fbc55b887"}
May 25 06:46:39.478773 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:39.478735    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:46:39.478975 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:39.478787    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:39.478975 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:39.478888    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 06:46:39.478975 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:39.478891    2572 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 06:46:39.478975 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:39.478939    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls podName:88e807ae-7d7e-426f-a3ab-25fa0d336a7f nodeName:}" failed. No retries permitted until 2026-05-25 06:46:43.478925504 +0000 UTC m=+41.715111874 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls") pod "dns-default-gz8v5" (UID: "88e807ae-7d7e-426f-a3ab-25fa0d336a7f") : secret "dns-default-metrics-tls" not found
May 25 06:46:39.478975 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:39.478952    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert podName:f1ba578b-9195-4d91-be43-38f7a5820121 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:43.478947018 +0000 UTC m=+41.715133388 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert") pod "ingress-canary-x4v6p" (UID: "f1ba578b-9195-4d91-be43-38f7a5820121") : secret "canary-serving-cert" not found
May 25 06:46:39.588168 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:39.588134    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5jwq4" event={"ID":"a6aeda4f-547c-488a-9414-e6ab811425c6","Type":"ContainerStarted","Data":"f94f5fef93dac58a0d955b73c8ed76402b747f53b61c5f7d11f49e4973833b70"}
May 25 06:46:39.608498 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:39.608450    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5jwq4" podStartSLOduration=4.707104122 podStartE2EDuration="37.608435231s" podCreationTimestamp="2026-05-25 06:46:02 +0000 UTC" firstStartedPulling="2026-05-25 06:46:03.507909789 +0000 UTC m=+1.744096174" lastFinishedPulling="2026-05-25 06:46:36.40924091 +0000 UTC m=+34.645427283" observedRunningTime="2026-05-25 06:46:39.60801882 +0000 UTC m=+37.844205213" watchObservedRunningTime="2026-05-25 06:46:39.608435231 +0000 UTC m=+37.844621623"
May 25 06:46:43.507963 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:43.507916    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:43.508350 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:43.507992    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:46:43.508350 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:43.508072    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 06:46:43.508350 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:43.508147    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls podName:88e807ae-7d7e-426f-a3ab-25fa0d336a7f nodeName:}" failed. No retries permitted until 2026-05-25 06:46:51.508128098 +0000 UTC m=+49.744314473 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls") pod "dns-default-gz8v5" (UID: "88e807ae-7d7e-426f-a3ab-25fa0d336a7f") : secret "dns-default-metrics-tls" not found
May 25 06:46:43.508350 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:43.508076    2572 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 06:46:43.508350 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:43.508208    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert podName:f1ba578b-9195-4d91-be43-38f7a5820121 nodeName:}" failed. No retries permitted until 2026-05-25 06:46:51.508195307 +0000 UTC m=+49.744381679 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert") pod "ingress-canary-x4v6p" (UID: "f1ba578b-9195-4d91-be43-38f7a5820121") : secret "canary-serving-cert" not found
May 25 06:46:51.567070 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:51.567028    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:46:51.567619 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:46:51.567096    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:46:51.567619 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:51.567186    2572 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 06:46:51.567619 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:51.567189    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 06:46:51.567619 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:51.567265    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls podName:88e807ae-7d7e-426f-a3ab-25fa0d336a7f nodeName:}" failed. No retries permitted until 2026-05-25 06:47:07.567247633 +0000 UTC m=+65.803434006 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls") pod "dns-default-gz8v5" (UID: "88e807ae-7d7e-426f-a3ab-25fa0d336a7f") : secret "dns-default-metrics-tls" not found
May 25 06:46:51.567619 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:46:51.567280    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert podName:f1ba578b-9195-4d91-be43-38f7a5820121 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:07.567274674 +0000 UTC m=+65.803461044 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert") pod "ingress-canary-x4v6p" (UID: "f1ba578b-9195-4d91-be43-38f7a5820121") : secret "canary-serving-cert" not found
May 25 06:47:01.571329 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:01.571294    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-nxvdb"
May 25 06:47:06.971976 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:06.971929    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:47:06.974246 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:06.974228    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 25 06:47:06.982885 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:06.982864    2572 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 25 06:47:06.982959 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:06.982935    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs podName:28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3 nodeName:}" failed. No retries permitted until 2026-05-25 06:48:10.982918104 +0000 UTC m=+129.219104478 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs") pod "network-metrics-daemon-lqsp4" (UID: "28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3") : secret "metrics-daemon-secret" not found
May 25 06:47:07.072786 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:07.072755    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-442st\" (UniqueName: \"kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st\") pod \"network-check-target-pttm7\" (UID: \"50545ca0-b117-4f61-8439-62bd12fbf0c0\") " pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:47:07.075478 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:07.075461    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 25 06:47:07.084882 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:07.084859    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 25 06:47:07.096108 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:07.096077    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-442st\" (UniqueName: \"kubernetes.io/projected/50545ca0-b117-4f61-8439-62bd12fbf0c0-kube-api-access-442st\") pod \"network-check-target-pttm7\" (UID: \"50545ca0-b117-4f61-8439-62bd12fbf0c0\") " pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:47:07.188812 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:07.188780    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-nmwpq\""
May 25 06:47:07.197514 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:07.197483    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:47:07.373742 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:07.373714    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-pttm7"]
May 25 06:47:07.377725 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:47:07.377693    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50545ca0_b117_4f61_8439_62bd12fbf0c0.slice/crio-be70e0b5ab2a15f07dd9d2012817ad9f0e0434ffd29653de897a7a0aaf6316d4 WatchSource:0}: Error finding container be70e0b5ab2a15f07dd9d2012817ad9f0e0434ffd29653de897a7a0aaf6316d4: Status 404 returned error can't find the container with id be70e0b5ab2a15f07dd9d2012817ad9f0e0434ffd29653de897a7a0aaf6316d4
May 25 06:47:07.577270 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:07.577234    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:47:07.577428 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:07.577282    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:47:07.577428 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:07.577374    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 06:47:07.577428 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:07.577376    2572 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 06:47:07.577524 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:07.577438    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert podName:f1ba578b-9195-4d91-be43-38f7a5820121 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:39.57742392 +0000 UTC m=+97.813610293 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert") pod "ingress-canary-x4v6p" (UID: "f1ba578b-9195-4d91-be43-38f7a5820121") : secret "canary-serving-cert" not found
May 25 06:47:07.577524 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:07.577452    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls podName:88e807ae-7d7e-426f-a3ab-25fa0d336a7f nodeName:}" failed. No retries permitted until 2026-05-25 06:47:39.577445732 +0000 UTC m=+97.813632102 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls") pod "dns-default-gz8v5" (UID: "88e807ae-7d7e-426f-a3ab-25fa0d336a7f") : secret "dns-default-metrics-tls" not found
May 25 06:47:07.637019 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:07.636983    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pttm7" event={"ID":"50545ca0-b117-4f61-8439-62bd12fbf0c0","Type":"ContainerStarted","Data":"be70e0b5ab2a15f07dd9d2012817ad9f0e0434ffd29653de897a7a0aaf6316d4"}
May 25 06:47:10.643390 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:10.643346    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-pttm7" event={"ID":"50545ca0-b117-4f61-8439-62bd12fbf0c0","Type":"ContainerStarted","Data":"8a652322cdb0b14560d54c3bc3384ea18bdb03275896aa9808c54b107f60944e"}
May 25 06:47:10.643783 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:10.643526    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:47:10.657788 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:10.657738    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-pttm7" podStartSLOduration=66.001482047 podStartE2EDuration="1m8.657723452s" podCreationTimestamp="2026-05-25 06:46:02 +0000 UTC" firstStartedPulling="2026-05-25 06:47:07.379513565 +0000 UTC m=+65.615699936" lastFinishedPulling="2026-05-25 06:47:10.035754955 +0000 UTC m=+68.271941341" observedRunningTime="2026-05-25 06:47:10.657323675 +0000 UTC m=+68.893510067" watchObservedRunningTime="2026-05-25 06:47:10.657723452 +0000 UTC m=+68.893909883"
May 25 06:47:39.587493 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:39.587454    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:47:39.587973 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:39.587503    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:47:39.587973 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:39.587597    2572 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 25 06:47:39.587973 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:39.587610    2572 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 25 06:47:39.587973 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:39.587651    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls podName:88e807ae-7d7e-426f-a3ab-25fa0d336a7f nodeName:}" failed. No retries permitted until 2026-05-25 06:48:43.587636717 +0000 UTC m=+161.823823086 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls") pod "dns-default-gz8v5" (UID: "88e807ae-7d7e-426f-a3ab-25fa0d336a7f") : secret "dns-default-metrics-tls" not found
May 25 06:47:39.587973 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:39.587691    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert podName:f1ba578b-9195-4d91-be43-38f7a5820121 nodeName:}" failed. No retries permitted until 2026-05-25 06:48:43.587673812 +0000 UTC m=+161.823860183 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert") pod "ingress-canary-x4v6p" (UID: "f1ba578b-9195-4d91-be43-38f7a5820121") : secret "canary-serving-cert" not found
May 25 06:47:41.647969 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:41.647943    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-pttm7"
May 25 06:47:46.836229 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.836195    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"]
May 25 06:47:46.838598 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.838576    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"]
May 25 06:47:46.838743 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.838722    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:47:46.840273 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.840254    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"
May 25 06:47:46.840698 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.840681    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"samples-operator-tls\""
May 25 06:47:46.840784 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.840766    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qkm8v\""
May 25 06:47:46.841111 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.841097    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"kube-root-ca.crt\""
May 25 06:47:46.841355 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.841344    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-samples-operator\"/\"openshift-service-ca.crt\""
May 25 06:47:46.841898 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.841881    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-t7rsr\""
May 25 06:47:46.841957 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.841935    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
May 25 06:47:46.842018 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.842000    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
May 25 06:47:46.842573 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.842550    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
May 25 06:47:46.843105 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.843092    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
May 25 06:47:46.848172 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.848151    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"]
May 25 06:47:46.860067 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.860044    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"]
May 25 06:47:46.938177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.938141    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8d2c6188-c2a6-43cd-a2ec-f7eb401bb061-config\") pod \"service-ca-operator-57b5b68b8-h9nz4\" (UID: \"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"
May 25 06:47:46.938373 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.938204    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zzp82\" (UniqueName: \"kubernetes.io/projected/8d2c6188-c2a6-43cd-a2ec-f7eb401bb061-kube-api-access-zzp82\") pod \"service-ca-operator-57b5b68b8-h9nz4\" (UID: \"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"
May 25 06:47:46.938373 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.938270    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mqwdc\" (UniqueName: \"kubernetes.io/projected/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-kube-api-access-mqwdc\") pod \"cluster-samples-operator-697587599-w5dzf\" (UID: \"c4147cb5-83db-48f2-8efd-ffc001b1d4c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:47:46.938373 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.938349    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-w5dzf\" (UID: \"c4147cb5-83db-48f2-8efd-ffc001b1d4c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:47:46.938527 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.938398    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d2c6188-c2a6-43cd-a2ec-f7eb401bb061-serving-cert\") pod \"service-ca-operator-57b5b68b8-h9nz4\" (UID: \"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"
May 25 06:47:46.939307 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.939281    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"]
May 25 06:47:46.941159 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.941141    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"]
May 25 06:47:46.941325 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.941295    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:46.942784 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.942766    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78ddd5c598-9msrl"]
May 25 06:47:46.942952 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.942936    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"
May 25 06:47:46.944132 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.944108    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-tls\""
May 25 06:47:46.944212 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.944119    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 25 06:47:46.944435 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.944416    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:46.944644 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.944621    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 25 06:47:46.944917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.944898    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ljwbm\""
May 25 06:47:46.944917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.944908    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemetry-config\""
May 25 06:47:46.945194 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.945181    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
May 25 06:47:46.946747 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.946717    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
May 25 06:47:46.946885 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.946825    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 25 06:47:46.946885 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.946861    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
May 25 06:47:46.946885 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.946870    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2gmjv\""
May 25 06:47:46.947101 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.946951    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-74qkl\""
May 25 06:47:46.947101 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.946984    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
May 25 06:47:46.947101 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.947078    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 25 06:47:46.947287 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.947154    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 25 06:47:46.952596 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.952575    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 25 06:47:46.956212 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.956174    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"]
May 25 06:47:46.957340 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.957323    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"]
May 25 06:47:46.958118 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:46.958093    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78ddd5c598-9msrl"]
May 25 06:47:47.038723 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.038682    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:47.038957 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.038740    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8af8432-7b6d-4e9c-acb6-fedaa77d523f-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-7vk74\" (UID: \"a8af8432-7b6d-4e9c-acb6-fedaa77d523f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"
May 25 06:47:47.038957 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.038773    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8d2c6188-c2a6-43cd-a2ec-f7eb401bb061-config\") pod \"service-ca-operator-57b5b68b8-h9nz4\" (UID: \"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"
May 25 06:47:47.038957 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.038794    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a8af8432-7b6d-4e9c-acb6-fedaa77d523f-config\") pod \"kube-storage-version-migrator-operator-565644f49-7vk74\" (UID: \"a8af8432-7b6d-4e9c-acb6-fedaa77d523f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"
May 25 06:47:47.038957 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.038852    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zzp82\" (UniqueName: \"kubernetes.io/projected/8d2c6188-c2a6-43cd-a2ec-f7eb401bb061-kube-api-access-zzp82\") pod \"service-ca-operator-57b5b68b8-h9nz4\" (UID: \"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"
May 25 06:47:47.038957 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.038878    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-ca-trust-extracted\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.038957 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.038911    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-image-registry-private-configuration\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.038957 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.038936    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-trusted-ca\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.039264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.038965    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mqwdc\" (UniqueName: \"kubernetes.io/projected/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-kube-api-access-mqwdc\") pod \"cluster-samples-operator-697587599-w5dzf\" (UID: \"c4147cb5-83db-48f2-8efd-ffc001b1d4c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:47:47.039264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.038991    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-installation-pull-secrets\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.039264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.039055    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ps28d\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-kube-api-access-ps28d\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.039264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.039099    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-w5dzf\" (UID: \"c4147cb5-83db-48f2-8efd-ffc001b1d4c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:47:47.039264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.039139    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-bound-sa-token\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.039264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.039212    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7cde0c58-8416-438c-ba27-4e5de27d3d91-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:47.039264 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.039242    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q6hlz\" (UniqueName: \"kubernetes.io/projected/7cde0c58-8416-438c-ba27-4e5de27d3d91-kube-api-access-q6hlz\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:47.039264 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.039259    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 25 06:47:47.039581 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.039305    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t5j2k\" (UniqueName: \"kubernetes.io/projected/a8af8432-7b6d-4e9c-acb6-fedaa77d523f-kube-api-access-t5j2k\") pod \"kube-storage-version-migrator-operator-565644f49-7vk74\" (UID: \"a8af8432-7b6d-4e9c-acb6-fedaa77d523f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"
May 25 06:47:47.039581 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.039317    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls podName:c4147cb5-83db-48f2-8efd-ffc001b1d4c4 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:47.539300192 +0000 UTC m=+105.775486566 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls") pod "cluster-samples-operator-697587599-w5dzf" (UID: "c4147cb5-83db-48f2-8efd-ffc001b1d4c4") : secret "samples-operator-tls" not found
May 25 06:47:47.039581 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.039315    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/8d2c6188-c2a6-43cd-a2ec-f7eb401bb061-config\") pod \"service-ca-operator-57b5b68b8-h9nz4\" (UID: \"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"
May 25 06:47:47.039581 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.039367    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d2c6188-c2a6-43cd-a2ec-f7eb401bb061-serving-cert\") pod \"service-ca-operator-57b5b68b8-h9nz4\" (UID: \"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"
May 25 06:47:47.039581 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.039394    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.039581 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.039426    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-certificates\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.041481 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.041462    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/8d2c6188-c2a6-43cd-a2ec-f7eb401bb061-serving-cert\") pod \"service-ca-operator-57b5b68b8-h9nz4\" (UID: \"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"
May 25 06:47:47.047290 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.047270    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zzp82\" (UniqueName: \"kubernetes.io/projected/8d2c6188-c2a6-43cd-a2ec-f7eb401bb061-kube-api-access-zzp82\") pod \"service-ca-operator-57b5b68b8-h9nz4\" (UID: \"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"
May 25 06:47:47.047458 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.047439    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mqwdc\" (UniqueName: \"kubernetes.io/projected/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-kube-api-access-mqwdc\") pod \"cluster-samples-operator-697587599-w5dzf\" (UID: \"c4147cb5-83db-48f2-8efd-ffc001b1d4c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:47:47.140220 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140116    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-ca-trust-extracted\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.140220 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140179    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-image-registry-private-configuration\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.140220 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140208    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-trusted-ca\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.140502 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140234    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-installation-pull-secrets\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.140502 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140267    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ps28d\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-kube-api-access-ps28d\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.140502 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140306    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-bound-sa-token\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.140502 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140328    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7cde0c58-8416-438c-ba27-4e5de27d3d91-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:47.140502 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140351    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q6hlz\" (UniqueName: \"kubernetes.io/projected/7cde0c58-8416-438c-ba27-4e5de27d3d91-kube-api-access-q6hlz\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:47.140502 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140380    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t5j2k\" (UniqueName: \"kubernetes.io/projected/a8af8432-7b6d-4e9c-acb6-fedaa77d523f-kube-api-access-t5j2k\") pod \"kube-storage-version-migrator-operator-565644f49-7vk74\" (UID: \"a8af8432-7b6d-4e9c-acb6-fedaa77d523f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"
May 25 06:47:47.140502 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140425    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.140502 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140451    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-certificates\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.140502 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140493    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:47.140956 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140535    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8af8432-7b6d-4e9c-acb6-fedaa77d523f-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-7vk74\" (UID: \"a8af8432-7b6d-4e9c-acb6-fedaa77d523f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"
May 25 06:47:47.140956 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140566    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a8af8432-7b6d-4e9c-acb6-fedaa77d523f-config\") pod \"kube-storage-version-migrator-operator-565644f49-7vk74\" (UID: \"a8af8432-7b6d-4e9c-acb6-fedaa77d523f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"
May 25 06:47:47.140956 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.140614    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-ca-trust-extracted\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.140956 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.140704    2572 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 06:47:47.140956 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.140720    2572 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ddd5c598-9msrl: secret "image-registry-tls" not found
May 25 06:47:47.140956 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.140784    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls podName:815efb61-c2de-4c40-a2f5-9f4f9a4737b3 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:47.640765532 +0000 UTC m=+105.876951915 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls") pod "image-registry-78ddd5c598-9msrl" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3") : secret "image-registry-tls" not found
May 25 06:47:47.141259 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.140960    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 25 06:47:47.141259 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.141016    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls podName:7cde0c58-8416-438c-ba27-4e5de27d3d91 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:47.640998922 +0000 UTC m=+105.877185303 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-4cncm" (UID: "7cde0c58-8416-438c-ba27-4e5de27d3d91") : secret "cluster-monitoring-operator-tls" not found
May 25 06:47:47.141259 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.141153    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemetry-config\" (UniqueName: \"kubernetes.io/configmap/7cde0c58-8416-438c-ba27-4e5de27d3d91-telemetry-config\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:47.141414 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.141391    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-trusted-ca\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.141504 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.141484    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/a8af8432-7b6d-4e9c-acb6-fedaa77d523f-config\") pod \"kube-storage-version-migrator-operator-565644f49-7vk74\" (UID: \"a8af8432-7b6d-4e9c-acb6-fedaa77d523f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"
May 25 06:47:47.141708 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.141685    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-certificates\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.143100 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.142873    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/a8af8432-7b6d-4e9c-acb6-fedaa77d523f-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-7vk74\" (UID: \"a8af8432-7b6d-4e9c-acb6-fedaa77d523f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"
May 25 06:47:47.143100 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.142910    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-image-registry-private-configuration\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.143238 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.143126    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-installation-pull-secrets\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.148893 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.148868    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t5j2k\" (UniqueName: \"kubernetes.io/projected/a8af8432-7b6d-4e9c-acb6-fedaa77d523f-kube-api-access-t5j2k\") pod \"kube-storage-version-migrator-operator-565644f49-7vk74\" (UID: \"a8af8432-7b6d-4e9c-acb6-fedaa77d523f\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"
May 25 06:47:47.149663 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.149630    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-bound-sa-token\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.149770 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.149689    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ps28d\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-kube-api-access-ps28d\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.149953 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.149928    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q6hlz\" (UniqueName: \"kubernetes.io/projected/7cde0c58-8416-438c-ba27-4e5de27d3d91-kube-api-access-q6hlz\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:47.154345 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.154327    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"
May 25 06:47:47.260529 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.260492    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"
May 25 06:47:47.277030 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.276979    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4"]
May 25 06:47:47.281068 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:47:47.281039    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8d2c6188_c2a6_43cd_a2ec_f7eb401bb061.slice/crio-aa70d3e8f023e68c5dba78df8754811504d277f42427548fa8bb69284a9073e1 WatchSource:0}: Error finding container aa70d3e8f023e68c5dba78df8754811504d277f42427548fa8bb69284a9073e1: Status 404 returned error can't find the container with id aa70d3e8f023e68c5dba78df8754811504d277f42427548fa8bb69284a9073e1
May 25 06:47:47.381554 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.381523    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74"]
May 25 06:47:47.385310 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:47:47.385280    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda8af8432_7b6d_4e9c_acb6_fedaa77d523f.slice/crio-8dd2fa164748cff9ba8d08337bbca6a70f98e5970ae359a2ddcea42b99f548c4 WatchSource:0}: Error finding container 8dd2fa164748cff9ba8d08337bbca6a70f98e5970ae359a2ddcea42b99f548c4: Status 404 returned error can't find the container with id 8dd2fa164748cff9ba8d08337bbca6a70f98e5970ae359a2ddcea42b99f548c4
May 25 06:47:47.543941 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.543833    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-w5dzf\" (UID: \"c4147cb5-83db-48f2-8efd-ffc001b1d4c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:47:47.544085 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.543986    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 25 06:47:47.544085 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.544053    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls podName:c4147cb5-83db-48f2-8efd-ffc001b1d4c4 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:48.544037154 +0000 UTC m=+106.780223528 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls") pod "cluster-samples-operator-697587599-w5dzf" (UID: "c4147cb5-83db-48f2-8efd-ffc001b1d4c4") : secret "samples-operator-tls" not found
May 25 06:47:47.645235 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.645194    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:47.645235 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.645240    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:47.645436 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.645336    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 25 06:47:47.645436 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.645346    2572 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 06:47:47.645436 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.645365    2572 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ddd5c598-9msrl: secret "image-registry-tls" not found
May 25 06:47:47.645436 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.645389    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls podName:7cde0c58-8416-438c-ba27-4e5de27d3d91 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:48.645376181 +0000 UTC m=+106.881562550 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-4cncm" (UID: "7cde0c58-8416-438c-ba27-4e5de27d3d91") : secret "cluster-monitoring-operator-tls" not found
May 25 06:47:47.645436 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:47.645409    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls podName:815efb61-c2de-4c40-a2f5-9f4f9a4737b3 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:48.645395262 +0000 UTC m=+106.881581636 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls") pod "image-registry-78ddd5c598-9msrl" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3") : secret "image-registry-tls" not found
May 25 06:47:47.718963 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.718921    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74" event={"ID":"a8af8432-7b6d-4e9c-acb6-fedaa77d523f","Type":"ContainerStarted","Data":"8dd2fa164748cff9ba8d08337bbca6a70f98e5970ae359a2ddcea42b99f548c4"}
May 25 06:47:47.719900 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:47.719878    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4" event={"ID":"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061","Type":"ContainerStarted","Data":"aa70d3e8f023e68c5dba78df8754811504d277f42427548fa8bb69284a9073e1"}
May 25 06:47:48.554369 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:48.554329    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-w5dzf\" (UID: \"c4147cb5-83db-48f2-8efd-ffc001b1d4c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:47:48.554812 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:48.554523    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 25 06:47:48.554812 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:48.554599    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls podName:c4147cb5-83db-48f2-8efd-ffc001b1d4c4 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:50.554577748 +0000 UTC m=+108.790764127 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls") pod "cluster-samples-operator-697587599-w5dzf" (UID: "c4147cb5-83db-48f2-8efd-ffc001b1d4c4") : secret "samples-operator-tls" not found
May 25 06:47:48.655806 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:48.655768    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:48.655983 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:48.655931    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:48.655983 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:48.655939    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 25 06:47:48.656068 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:48.656005    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls podName:7cde0c58-8416-438c-ba27-4e5de27d3d91 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:50.655986601 +0000 UTC m=+108.892172988 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-4cncm" (UID: "7cde0c58-8416-438c-ba27-4e5de27d3d91") : secret "cluster-monitoring-operator-tls" not found
May 25 06:47:48.656068 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:48.656036    2572 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 06:47:48.656068 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:48.656048    2572 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ddd5c598-9msrl: secret "image-registry-tls" not found
May 25 06:47:48.656189 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:48.656082    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls podName:815efb61-c2de-4c40-a2f5-9f4f9a4737b3 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:50.656070581 +0000 UTC m=+108.892256957 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls") pod "image-registry-78ddd5c598-9msrl" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3") : secret "image-registry-tls" not found
May 25 06:47:50.572027 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.571986    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-w5dzf\" (UID: \"c4147cb5-83db-48f2-8efd-ffc001b1d4c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:47:50.572509 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:50.572153    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 25 06:47:50.572509 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:50.572244    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls podName:c4147cb5-83db-48f2-8efd-ffc001b1d4c4 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:54.572221908 +0000 UTC m=+112.808408278 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls") pod "cluster-samples-operator-697587599-w5dzf" (UID: "c4147cb5-83db-48f2-8efd-ffc001b1d4c4") : secret "samples-operator-tls" not found
May 25 06:47:50.672519 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.672488    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:50.672710 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.672530    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:50.672710 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:50.672626    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 25 06:47:50.672710 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:50.672637    2572 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 06:47:50.672710 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:50.672657    2572 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ddd5c598-9msrl: secret "image-registry-tls" not found
May 25 06:47:50.672710 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:50.672680    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls podName:7cde0c58-8416-438c-ba27-4e5de27d3d91 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:54.672666728 +0000 UTC m=+112.908853099 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-4cncm" (UID: "7cde0c58-8416-438c-ba27-4e5de27d3d91") : secret "cluster-monitoring-operator-tls" not found
May 25 06:47:50.672898 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:50.672729    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls podName:815efb61-c2de-4c40-a2f5-9f4f9a4737b3 nodeName:}" failed. No retries permitted until 2026-05-25 06:47:54.672713047 +0000 UTC m=+112.908899421 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls") pod "image-registry-78ddd5c598-9msrl" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3") : secret "image-registry-tls" not found
May 25 06:47:50.727296 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.727256    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74" event={"ID":"a8af8432-7b6d-4e9c-acb6-fedaa77d523f","Type":"ContainerStarted","Data":"36570cec4479f32d6e7a0dcd892e52a12b7b552aecbf968cefc892f7d97cf8ca"}
May 25 06:47:50.728331 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.728306    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4" event={"ID":"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061","Type":"ContainerStarted","Data":"730af3ceb93cbf0f222ecafd5ffe68825c1ff5f6571fe2ce93075a6f8e27bb1e"}
May 25 06:47:50.741362 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.741317    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74" podStartSLOduration=2.255987303 podStartE2EDuration="4.741302466s" podCreationTimestamp="2026-05-25 06:47:46 +0000 UTC" firstStartedPulling="2026-05-25 06:47:47.387107605 +0000 UTC m=+105.623293975" lastFinishedPulling="2026-05-25 06:47:49.872422752 +0000 UTC m=+108.108609138" observedRunningTime="2026-05-25 06:47:50.74099247 +0000 UTC m=+108.977178875" watchObservedRunningTime="2026-05-25 06:47:50.741302466 +0000 UTC m=+108.977488835"
May 25 06:47:50.753415 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.753360    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4" podStartSLOduration=2.167607808 podStartE2EDuration="4.753345333s" podCreationTimestamp="2026-05-25 06:47:46 +0000 UTC" firstStartedPulling="2026-05-25 06:47:47.283300994 +0000 UTC m=+105.519487365" lastFinishedPulling="2026-05-25 06:47:49.869038509 +0000 UTC m=+108.105224890" observedRunningTime="2026-05-25 06:47:50.752549189 +0000 UTC m=+108.988735582" watchObservedRunningTime="2026-05-25 06:47:50.753345333 +0000 UTC m=+108.989531725"
May 25 06:47:50.857104 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.857021    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-wk9ms"]
May 25 06:47:50.860204 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.860187    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-wk9ms"
May 25 06:47:50.862072 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.862050    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"openshift-service-ca.crt\""
May 25 06:47:50.862181 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.862136    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-root-ca.crt\""
May 25 06:47:50.862232 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.862194    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator\"/\"kube-storage-version-migrator-sa-dockercfg-th5lf\""
May 25 06:47:50.867441 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.867404    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-wk9ms"]
May 25 06:47:50.975404 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:50.975376    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-952f4\" (UniqueName: \"kubernetes.io/projected/05ee7237-b28a-4df8-b33e-5adc887b50ad-kube-api-access-952f4\") pod \"migrator-b985f5d89-wk9ms\" (UID: \"05ee7237-b28a-4df8-b33e-5adc887b50ad\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-wk9ms"
May 25 06:47:51.076208 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:51.076172    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-952f4\" (UniqueName: \"kubernetes.io/projected/05ee7237-b28a-4df8-b33e-5adc887b50ad-kube-api-access-952f4\") pod \"migrator-b985f5d89-wk9ms\" (UID: \"05ee7237-b28a-4df8-b33e-5adc887b50ad\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-wk9ms"
May 25 06:47:51.084007 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:51.083989    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-952f4\" (UniqueName: \"kubernetes.io/projected/05ee7237-b28a-4df8-b33e-5adc887b50ad-kube-api-access-952f4\") pod \"migrator-b985f5d89-wk9ms\" (UID: \"05ee7237-b28a-4df8-b33e-5adc887b50ad\") " pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-wk9ms"
May 25 06:47:51.170793 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:51.170703    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-wk9ms"
May 25 06:47:51.301631 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:51.301601    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator/migrator-b985f5d89-wk9ms"]
May 25 06:47:51.305071 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:47:51.305043    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod05ee7237_b28a_4df8_b33e_5adc887b50ad.slice/crio-40d178ee92cf45fc89be5fd3144f28ebbb67a44dac49357bd3340a37fb7830ce WatchSource:0}: Error finding container 40d178ee92cf45fc89be5fd3144f28ebbb67a44dac49357bd3340a37fb7830ce: Status 404 returned error can't find the container with id 40d178ee92cf45fc89be5fd3144f28ebbb67a44dac49357bd3340a37fb7830ce
May 25 06:47:51.730615 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:51.730581    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-wk9ms" event={"ID":"05ee7237-b28a-4df8-b33e-5adc887b50ad","Type":"ContainerStarted","Data":"40d178ee92cf45fc89be5fd3144f28ebbb67a44dac49357bd3340a37fb7830ce"}
May 25 06:47:52.736240 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:52.736202    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-wk9ms" event={"ID":"05ee7237-b28a-4df8-b33e-5adc887b50ad","Type":"ContainerStarted","Data":"9ba909092e80a2f35972eb194a17f9025c615afede934cf01ec70b396c49ffd0"}
May 25 06:47:52.736240 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:52.736240    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-wk9ms" event={"ID":"05ee7237-b28a-4df8-b33e-5adc887b50ad","Type":"ContainerStarted","Data":"f42fad071c9d27af6f4d0437a74895488685851c4d5df485269e9acc59ea3259"}
May 25 06:47:52.749826 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:52.749755    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator/migrator-b985f5d89-wk9ms" podStartSLOduration=1.5277377859999999 podStartE2EDuration="2.749739334s" podCreationTimestamp="2026-05-25 06:47:50 +0000 UTC" firstStartedPulling="2026-05-25 06:47:51.307130507 +0000 UTC m=+109.543316877" lastFinishedPulling="2026-05-25 06:47:52.529132053 +0000 UTC m=+110.765318425" observedRunningTime="2026-05-25 06:47:52.749223991 +0000 UTC m=+110.985410383" watchObservedRunningTime="2026-05-25 06:47:52.749739334 +0000 UTC m=+110.985925726"
May 25 06:47:53.952194 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:53.952166    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5gkzs_a569a74b-5db9-428c-8a4e-9692a5636e5d/dns-node-resolver/0.log"
May 25 06:47:54.605809 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:54.605765    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-w5dzf\" (UID: \"c4147cb5-83db-48f2-8efd-ffc001b1d4c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:47:54.606015 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:54.605945    2572 secret.go:189] Couldn't get secret openshift-cluster-samples-operator/samples-operator-tls: secret "samples-operator-tls" not found
May 25 06:47:54.606066 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:54.606015    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls podName:c4147cb5-83db-48f2-8efd-ffc001b1d4c4 nodeName:}" failed. No retries permitted until 2026-05-25 06:48:02.605999294 +0000 UTC m=+120.842185664 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "samples-operator-tls" (UniqueName: "kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls") pod "cluster-samples-operator-697587599-w5dzf" (UID: "c4147cb5-83db-48f2-8efd-ffc001b1d4c4") : secret "samples-operator-tls" not found
May 25 06:47:54.706452 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:54.706415    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:47:54.706652 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:54.706519    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:47:54.706652 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:54.706597    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 25 06:47:54.706771 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:54.706674    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls podName:7cde0c58-8416-438c-ba27-4e5de27d3d91 nodeName:}" failed. No retries permitted until 2026-05-25 06:48:02.706654377 +0000 UTC m=+120.942840761 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-4cncm" (UID: "7cde0c58-8416-438c-ba27-4e5de27d3d91") : secret "cluster-monitoring-operator-tls" not found
May 25 06:47:54.706771 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:54.706605    2572 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 25 06:47:54.706771 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:54.706698    2572 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-78ddd5c598-9msrl: secret "image-registry-tls" not found
May 25 06:47:54.706771 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:47:54.706738    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls podName:815efb61-c2de-4c40-a2f5-9f4f9a4737b3 nodeName:}" failed. No retries permitted until 2026-05-25 06:48:02.706726782 +0000 UTC m=+120.942913151 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls") pod "image-registry-78ddd5c598-9msrl" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3") : secret "image-registry-tls" not found
May 25 06:47:55.152827 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:55.152797    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-87m8j_c3130f84-1146-48b3-97c0-bf676f8eb29d/node-ca/0.log"
May 25 06:47:56.352096 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:56.352065    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-wk9ms_05ee7237-b28a-4df8-b33e-5adc887b50ad/migrator/0.log"
May 25 06:47:56.552004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:56.551970    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-wk9ms_05ee7237-b28a-4df8-b33e-5adc887b50ad/graceful-termination/0.log"
May 25 06:47:56.753373 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:47:56.753296    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-7vk74_a8af8432-7b6d-4e9c-acb6-fedaa77d523f/kube-storage-version-migrator-operator/0.log"
May 25 06:48:02.674407 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:02.674351    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-w5dzf\" (UID: \"c4147cb5-83db-48f2-8efd-ffc001b1d4c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:48:02.676861 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:02.676824    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"samples-operator-tls\" (UniqueName: \"kubernetes.io/secret/c4147cb5-83db-48f2-8efd-ffc001b1d4c4-samples-operator-tls\") pod \"cluster-samples-operator-697587599-w5dzf\" (UID: \"c4147cb5-83db-48f2-8efd-ffc001b1d4c4\") " pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:48:02.750000 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:02.749965    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-samples-operator\"/\"cluster-samples-operator-dockercfg-qkm8v\""
May 25 06:48:02.758742 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:02.758715    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"
May 25 06:48:02.775775 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:02.775731    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:48:02.775922 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:48:02.775908    2572 secret.go:189] Couldn't get secret openshift-monitoring/cluster-monitoring-operator-tls: secret "cluster-monitoring-operator-tls" not found
May 25 06:48:02.775981 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:02.775962    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:48:02.775981 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:48:02.775974    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls podName:7cde0c58-8416-438c-ba27-4e5de27d3d91 nodeName:}" failed. No retries permitted until 2026-05-25 06:48:18.775956669 +0000 UTC m=+137.012143047 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cluster-monitoring-operator-tls" (UniqueName: "kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls") pod "cluster-monitoring-operator-6c75959455-4cncm" (UID: "7cde0c58-8416-438c-ba27-4e5de27d3d91") : secret "cluster-monitoring-operator-tls" not found
May 25 06:48:02.778453 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:02.778428    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls\") pod \"image-registry-78ddd5c598-9msrl\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") " pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:48:02.867335 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:02.867301    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-2gmjv\""
May 25 06:48:02.875835 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:02.875816    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:48:02.880740 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:02.880710    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf"]
May 25 06:48:03.001387 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:03.001359    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78ddd5c598-9msrl"]
May 25 06:48:03.005034 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:03.005007    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod815efb61_c2de_4c40_a2f5_9f4f9a4737b3.slice/crio-7d140f33e86b296655ea9dc9ec690508c39d2762bdd77781b46dab53720532a7 WatchSource:0}: Error finding container 7d140f33e86b296655ea9dc9ec690508c39d2762bdd77781b46dab53720532a7: Status 404 returned error can't find the container with id 7d140f33e86b296655ea9dc9ec690508c39d2762bdd77781b46dab53720532a7
May 25 06:48:03.762235 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:03.762196    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf" event={"ID":"c4147cb5-83db-48f2-8efd-ffc001b1d4c4","Type":"ContainerStarted","Data":"768149a5b21bf80ba250a62ad4ca72ef47fdcffc57992ea16de40fa280759b97"}
May 25 06:48:03.763944 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:03.763907    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl" event={"ID":"815efb61-c2de-4c40-a2f5-9f4f9a4737b3","Type":"ContainerStarted","Data":"fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8"}
May 25 06:48:03.763944 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:03.763946    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl" event={"ID":"815efb61-c2de-4c40-a2f5-9f4f9a4737b3","Type":"ContainerStarted","Data":"7d140f33e86b296655ea9dc9ec690508c39d2762bdd77781b46dab53720532a7"}
May 25 06:48:03.764135 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:03.764054    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:48:03.781526 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:03.781133    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl" podStartSLOduration=17.781115855 podStartE2EDuration="17.781115855s" podCreationTimestamp="2026-05-25 06:47:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 06:48:03.780495473 +0000 UTC m=+122.016681866" watchObservedRunningTime="2026-05-25 06:48:03.781115855 +0000 UTC m=+122.017302259"
May 25 06:48:05.771018 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:05.770982    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf" event={"ID":"c4147cb5-83db-48f2-8efd-ffc001b1d4c4","Type":"ContainerStarted","Data":"68a1dbcffe2af1717c7433beafc534044647ad5784bb45adcdf0a46d2925ce68"}
May 25 06:48:05.771018 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:05.771022    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf" event={"ID":"c4147cb5-83db-48f2-8efd-ffc001b1d4c4","Type":"ContainerStarted","Data":"9e1f28cf1e1c454e2c1e58825977d676f5c58d32e2ed77c97e903f5504e3eb37"}
May 25 06:48:05.786973 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:05.786918    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-samples-operator/cluster-samples-operator-697587599-w5dzf" podStartSLOduration=17.872103321 podStartE2EDuration="19.786902249s" podCreationTimestamp="2026-05-25 06:47:46 +0000 UTC" firstStartedPulling="2026-05-25 06:48:02.926772442 +0000 UTC m=+121.162958816" lastFinishedPulling="2026-05-25 06:48:04.841571374 +0000 UTC m=+123.077757744" observedRunningTime="2026-05-25 06:48:05.785888978 +0000 UTC m=+124.022075370" watchObservedRunningTime="2026-05-25 06:48:05.786902249 +0000 UTC m=+124.023088641"
May 25 06:48:11.039798 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:11.039755    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:48:11.042136 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:11.042105    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3-metrics-certs\") pod \"network-metrics-daemon-lqsp4\" (UID: \"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3\") " pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:48:11.188795 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:11.188765    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-t49r8\""
May 25 06:48:11.197369 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:11.197348    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-lqsp4"
May 25 06:48:11.318190 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:11.318162    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-lqsp4"]
May 25 06:48:11.321504 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:11.321467    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod28839bdc_f2b5_4a95_bc4e_4a2ab9bab5e3.slice/crio-6981bea0ce598a102a034e5d73e1c29ed4c6c3d5645336d6240001b441e3de33 WatchSource:0}: Error finding container 6981bea0ce598a102a034e5d73e1c29ed4c6c3d5645336d6240001b441e3de33: Status 404 returned error can't find the container with id 6981bea0ce598a102a034e5d73e1c29ed4c6c3d5645336d6240001b441e3de33
May 25 06:48:11.785787 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:11.785696    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqsp4" event={"ID":"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3","Type":"ContainerStarted","Data":"6981bea0ce598a102a034e5d73e1c29ed4c6c3d5645336d6240001b441e3de33"}
May 25 06:48:12.789005 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:12.788968    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqsp4" event={"ID":"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3","Type":"ContainerStarted","Data":"2e73995c027c783ef60d1db9210b4ddf28f14a2f8947c34eced3ba2003486b92"}
May 25 06:48:12.789378 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:12.789012    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-lqsp4" event={"ID":"28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3","Type":"ContainerStarted","Data":"3f97014d1f545e99bb24c4f94b2f04425a225c41c14012e423161f49accbaad3"}
May 25 06:48:12.803821 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:12.803773    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-lqsp4" podStartSLOduration=129.957956829 podStartE2EDuration="2m10.80375629s" podCreationTimestamp="2026-05-25 06:46:02 +0000 UTC" firstStartedPulling="2026-05-25 06:48:11.323422852 +0000 UTC m=+129.559609237" lastFinishedPulling="2026-05-25 06:48:12.169222319 +0000 UTC m=+130.405408698" observedRunningTime="2026-05-25 06:48:12.802661469 +0000 UTC m=+131.038847862" watchObservedRunningTime="2026-05-25 06:48:12.80375629 +0000 UTC m=+131.039942682"
May 25 06:48:14.034479 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.034439    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6"]
May 25 06:48:14.037710 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.037691    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6"
May 25 06:48:14.039640 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.039616    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"default-dockercfg-ngtm5\""
May 25 06:48:14.040010 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.039995    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-console\"/\"networking-console-plugin-cert\""
May 25 06:48:14.040086 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.040069    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-console\"/\"networking-console-plugin\""
May 25 06:48:14.048491 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.048464    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6"]
May 25 06:48:14.065055 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.065027    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78ddd5c598-9msrl"]
May 25 06:48:14.094709 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.094673    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-78494c4cc8-g2vcq"]
May 25 06:48:14.097804 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.097783    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.105309 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.105281    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-jv7ph"]
May 25 06:48:14.108360 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.108335    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78494c4cc8-g2vcq"]
May 25 06:48:14.108495 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.108480    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.110731 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.110704    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 25 06:48:14.110835 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.110805    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 25 06:48:14.111033 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.111017    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 25 06:48:14.111321 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.111307    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-ql8fr\""
May 25 06:48:14.111577 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.111559    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 25 06:48:14.124596 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.124564    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jv7ph"]
May 25 06:48:14.166160 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.166132    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-lg6d6\" (UID: \"02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6"
May 25 06:48:14.166160 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.166168    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-lg6d6\" (UID: \"02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6"
May 25 06:48:14.266952 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.266910    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-registry-certificates\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.267127 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.266963    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-trusted-ca\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.267127 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267020    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-bound-sa-token\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.267127 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267044    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-installation-pull-secrets\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.267127 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267068    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa459401-1adc-422c-b88d-1965967d9eba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.267127 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267090    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-hqmbl\" (UniqueName: \"kubernetes.io/projected/aa459401-1adc-422c-b88d-1965967d9eba-kube-api-access-hqmbl\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.267281 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267161    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-image-registry-private-configuration\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.267281 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267184    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa459401-1adc-422c-b88d-1965967d9eba-crio-socket\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.267281 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267207    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-ca-trust-extracted\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.267281 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267221    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa459401-1adc-422c-b88d-1965967d9eba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.267281 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267268    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-lg6d6\" (UID: \"02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6"
May 25 06:48:14.267423 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267289    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-registry-tls\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.267423 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267311    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vlwkp\" (UniqueName: \"kubernetes.io/projected/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-kube-api-access-vlwkp\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.267423 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267344    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-lg6d6\" (UID: \"02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6"
May 25 06:48:14.267423 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267367    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa459401-1adc-422c-b88d-1965967d9eba-data-volume\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.268005 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.267986    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"nginx-conf\" (UniqueName: \"kubernetes.io/configmap/02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab-nginx-conf\") pod \"networking-console-plugin-d4d8f94f9-lg6d6\" (UID: \"02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6"
May 25 06:48:14.269914 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.269882    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"networking-console-plugin-cert\" (UniqueName: \"kubernetes.io/secret/02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab-networking-console-plugin-cert\") pod \"networking-console-plugin-d4d8f94f9-lg6d6\" (UID: \"02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab\") " pod="openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6"
May 25 06:48:14.346661 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.346618    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6"
May 25 06:48:14.368668 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.368639    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa459401-1adc-422c-b88d-1965967d9eba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.368833 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.368678    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-hqmbl\" (UniqueName: \"kubernetes.io/projected/aa459401-1adc-422c-b88d-1965967d9eba-kube-api-access-hqmbl\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.368833 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.368722    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-image-registry-private-configuration\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.368833 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.368742    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa459401-1adc-422c-b88d-1965967d9eba-crio-socket\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.369028 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.368822    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-ca-trust-extracted\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.369028 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.368948    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/aa459401-1adc-422c-b88d-1965967d9eba-crio-socket\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.369028 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.368991    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa459401-1adc-422c-b88d-1965967d9eba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.369188 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.369036    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-registry-tls\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.369188 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.369065    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vlwkp\" (UniqueName: \"kubernetes.io/projected/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-kube-api-access-vlwkp\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.369188 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.369094    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa459401-1adc-422c-b88d-1965967d9eba-data-volume\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.369188 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.369131    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-registry-certificates\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.369188 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.369175    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-trusted-ca\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.369432 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.369212    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-bound-sa-token\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.369432 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.369247    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-installation-pull-secrets\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.369432 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.369405    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-ca-trust-extracted\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.369628 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.369605    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/aa459401-1adc-422c-b88d-1965967d9eba-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.370042 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.370022    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/aa459401-1adc-422c-b88d-1965967d9eba-data-volume\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.370192 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.370125    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-registry-certificates\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.371173 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.370927    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-trusted-ca\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.371549 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.371530    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-image-registry-private-configuration\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.371642 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.371618    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/aa459401-1adc-422c-b88d-1965967d9eba-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.371911 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.371894    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-installation-pull-secrets\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.372256 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.372235    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-registry-tls\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.375473 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.375448    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-hqmbl\" (UniqueName: \"kubernetes.io/projected/aa459401-1adc-422c-b88d-1965967d9eba-kube-api-access-hqmbl\") pod \"insights-runtime-extractor-jv7ph\" (UID: \"aa459401-1adc-422c-b88d-1965967d9eba\") " pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.375837 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.375816    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vlwkp\" (UniqueName: \"kubernetes.io/projected/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-kube-api-access-vlwkp\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.379274 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.379250    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/ebfad36b-e93a-41b2-8a88-94d1352d3f7d-bound-sa-token\") pod \"image-registry-78494c4cc8-g2vcq\" (UID: \"ebfad36b-e93a-41b2-8a88-94d1352d3f7d\") " pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.406606 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.406540    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.417362 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.417336    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-jv7ph"
May 25 06:48:14.490932 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.490902    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6"]
May 25 06:48:14.494215 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:14.494169    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod02cc22a1_cdc8_4c2b_8a2c_1a5b5a830bab.slice/crio-c619cba35d8fa220dbdb140c0726aec6e55d8c86b687ed9d9a1cb3432e0867c8 WatchSource:0}: Error finding container c619cba35d8fa220dbdb140c0726aec6e55d8c86b687ed9d9a1cb3432e0867c8: Status 404 returned error can't find the container with id c619cba35d8fa220dbdb140c0726aec6e55d8c86b687ed9d9a1cb3432e0867c8
May 25 06:48:14.543685 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.543652    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-78494c4cc8-g2vcq"]
May 25 06:48:14.546668 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:14.546643    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebfad36b_e93a_41b2_8a88_94d1352d3f7d.slice/crio-0831d942e8e1085df20d3a28db59bda07f2f2ea26fc2500b7ef1411f9d0bdb12 WatchSource:0}: Error finding container 0831d942e8e1085df20d3a28db59bda07f2f2ea26fc2500b7ef1411f9d0bdb12: Status 404 returned error can't find the container with id 0831d942e8e1085df20d3a28db59bda07f2f2ea26fc2500b7ef1411f9d0bdb12
May 25 06:48:14.561896 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.561866    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-jv7ph"]
May 25 06:48:14.564963 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:14.564937    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa459401_1adc_422c_b88d_1965967d9eba.slice/crio-c02718f1f6d41f9d87254f3107ec4c61867256c9b14318c6993b2d44eef777c0 WatchSource:0}: Error finding container c02718f1f6d41f9d87254f3107ec4c61867256c9b14318c6993b2d44eef777c0: Status 404 returned error can't find the container with id c02718f1f6d41f9d87254f3107ec4c61867256c9b14318c6993b2d44eef777c0
May 25 06:48:14.796257 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.796220    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq" event={"ID":"ebfad36b-e93a-41b2-8a88-94d1352d3f7d","Type":"ContainerStarted","Data":"29e60d0909602283d040802467e9937538ca3584c7852df96d171fc5a234eeb3"}
May 25 06:48:14.796257 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.796264    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq" event={"ID":"ebfad36b-e93a-41b2-8a88-94d1352d3f7d","Type":"ContainerStarted","Data":"0831d942e8e1085df20d3a28db59bda07f2f2ea26fc2500b7ef1411f9d0bdb12"}
May 25 06:48:14.796489 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.796282    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:14.797576 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.797552    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jv7ph" event={"ID":"aa459401-1adc-422c-b88d-1965967d9eba","Type":"ContainerStarted","Data":"5d9852cb56117d84db7da03443b469a2ef78e42ca31c594bfaea5461a6183e6e"}
May 25 06:48:14.797576 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.797579    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jv7ph" event={"ID":"aa459401-1adc-422c-b88d-1965967d9eba","Type":"ContainerStarted","Data":"c02718f1f6d41f9d87254f3107ec4c61867256c9b14318c6993b2d44eef777c0"}
May 25 06:48:14.798450 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.798430    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6" event={"ID":"02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab","Type":"ContainerStarted","Data":"c619cba35d8fa220dbdb140c0726aec6e55d8c86b687ed9d9a1cb3432e0867c8"}
May 25 06:48:14.812889 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:14.812833    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq" podStartSLOduration=0.812819874 podStartE2EDuration="812.819874ms" podCreationTimestamp="2026-05-25 06:48:14 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 06:48:14.812466906 +0000 UTC m=+133.048653299" watchObservedRunningTime="2026-05-25 06:48:14.812819874 +0000 UTC m=+133.049006266"
May 25 06:48:15.802969 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:15.802927    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jv7ph" event={"ID":"aa459401-1adc-422c-b88d-1965967d9eba","Type":"ContainerStarted","Data":"381bd93637c5cfb63ebd4eede5e1cfc1ae1c7ae76e14390d08346aa008efb526"}
May 25 06:48:15.804215 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:15.804190    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6" event={"ID":"02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab","Type":"ContainerStarted","Data":"442512d266a21fc17f645aa49caa28b81e2ade83bfd38b3f417fbfa07963b501"}
May 25 06:48:15.818642 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:15.818593    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-console/networking-console-plugin-d4d8f94f9-lg6d6" podStartSLOduration=0.663724714 podStartE2EDuration="1.818546795s" podCreationTimestamp="2026-05-25 06:48:14 +0000 UTC" firstStartedPulling="2026-05-25 06:48:14.496812768 +0000 UTC m=+132.732999138" lastFinishedPulling="2026-05-25 06:48:15.651634832 +0000 UTC m=+133.887821219" observedRunningTime="2026-05-25 06:48:15.817755867 +0000 UTC m=+134.053942259" watchObservedRunningTime="2026-05-25 06:48:15.818546795 +0000 UTC m=+134.054733181"
May 25 06:48:17.811061 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:17.811022    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-jv7ph" event={"ID":"aa459401-1adc-422c-b88d-1965967d9eba","Type":"ContainerStarted","Data":"4d413ab565bfec46543c2947b9148a9778b09c7cfbde11a06a431fe084d7fbfa"}
May 25 06:48:17.827172 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:17.827112    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-jv7ph" podStartSLOduration=1.6166970219999999 podStartE2EDuration="3.827090324s" podCreationTimestamp="2026-05-25 06:48:14 +0000 UTC" firstStartedPulling="2026-05-25 06:48:14.618869118 +0000 UTC m=+132.855055487" lastFinishedPulling="2026-05-25 06:48:16.829262415 +0000 UTC m=+135.065448789" observedRunningTime="2026-05-25 06:48:17.826095218 +0000 UTC m=+136.062281621" watchObservedRunningTime="2026-05-25 06:48:17.827090324 +0000 UTC m=+136.063276717"
May 25 06:48:18.808269 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:18.808230    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:48:18.810834 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:18.810809    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-monitoring-operator-tls\" (UniqueName: \"kubernetes.io/secret/7cde0c58-8416-438c-ba27-4e5de27d3d91-cluster-monitoring-operator-tls\") pod \"cluster-monitoring-operator-6c75959455-4cncm\" (UID: \"7cde0c58-8416-438c-ba27-4e5de27d3d91\") " pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:48:19.055381 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:19.055349    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"cluster-monitoring-operator-dockercfg-ljwbm\""
May 25 06:48:19.064008 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:19.063949    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"
May 25 06:48:19.201539 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:19.201473    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm"]
May 25 06:48:19.205279 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:19.205248    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cde0c58_8416_438c_ba27_4e5de27d3d91.slice/crio-c62f0dbb6ed2890ae39c878a9ffc19d6d8495b3088bc9ed8af8693a252ee610c WatchSource:0}: Error finding container c62f0dbb6ed2890ae39c878a9ffc19d6d8495b3088bc9ed8af8693a252ee610c: Status 404 returned error can't find the container with id c62f0dbb6ed2890ae39c878a9ffc19d6d8495b3088bc9ed8af8693a252ee610c
May 25 06:48:19.821154 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:19.821115    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm" event={"ID":"7cde0c58-8416-438c-ba27-4e5de27d3d91","Type":"ContainerStarted","Data":"c62f0dbb6ed2890ae39c878a9ffc19d6d8495b3088bc9ed8af8693a252ee610c"}
May 25 06:48:21.829728 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:21.829690    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm" event={"ID":"7cde0c58-8416-438c-ba27-4e5de27d3d91","Type":"ContainerStarted","Data":"de5526ab9ef6323b48dd9ee5c530c80c49e2bfaadfd7a1ec887b675885cc4781"}
May 25 06:48:21.844192 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:21.844144    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/cluster-monitoring-operator-6c75959455-4cncm" podStartSLOduration=33.839057551 podStartE2EDuration="35.844130245s" podCreationTimestamp="2026-05-25 06:47:46 +0000 UTC" firstStartedPulling="2026-05-25 06:48:19.207137866 +0000 UTC m=+137.443324236" lastFinishedPulling="2026-05-25 06:48:21.212210545 +0000 UTC m=+139.448396930" observedRunningTime="2026-05-25 06:48:21.843218784 +0000 UTC m=+140.079405176" watchObservedRunningTime="2026-05-25 06:48:21.844130245 +0000 UTC m=+140.080316636"
May 25 06:48:24.070606 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:24.070558    2572 patch_prober.go:28] interesting pod/image-registry-78ddd5c598-9msrl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 25 06:48:24.071056 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:24.070627    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl" podUID="815efb61-c2de-4c40-a2f5-9f4f9a4737b3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 25 06:48:28.114556 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.114511    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"]
May 25 06:48:28.117431 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.117415    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.120003 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.119978    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
May 25 06:48:28.120003 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.119996    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
May 25 06:48:28.120186 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.120018    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 25 06:48:28.120186 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.119984    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-xfxsm\""
May 25 06:48:28.130169 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.130135    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"]
May 25 06:48:28.143411 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.143384    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-cwf8g"]
May 25 06:48:28.145451 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.145433    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.147789 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.147770    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 25 06:48:28.147950 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.147934    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 25 06:48:28.148035 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.148021    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 25 06:48:28.148107 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.148089    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-v5pgb\""
May 25 06:48:28.182007 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.181972    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97787552-67e5-4f37-a9e9-8f53b7d66ca1-metrics-client-ca\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.182163 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.182018    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-accelerators-collector-config\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.182163 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.182080    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-wtmp\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.182163 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.182124    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97787552-67e5-4f37-a9e9-8f53b7d66ca1-root\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.182163 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.182153    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/369c0515-c0a1-4042-bcd9-8efbe2663bd8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-nwb6h\" (UID: \"369c0515-c0a1-4042-bcd9-8efbe2663bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.182350 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.182209    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-tls\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.182350 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.182233    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.182350 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.182288    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97787552-67e5-4f37-a9e9-8f53b7d66ca1-sys\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.182350 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.182317    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-textfile\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.182503 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.182364    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/369c0515-c0a1-4042-bcd9-8efbe2663bd8-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-nwb6h\" (UID: \"369c0515-c0a1-4042-bcd9-8efbe2663bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.182503 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.182397    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7hq4z\" (UniqueName: \"kubernetes.io/projected/97787552-67e5-4f37-a9e9-8f53b7d66ca1-kube-api-access-7hq4z\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.182503 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.182429    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zk76z\" (UniqueName: \"kubernetes.io/projected/369c0515-c0a1-4042-bcd9-8efbe2663bd8-kube-api-access-zk76z\") pod \"openshift-state-metrics-9c959559f-nwb6h\" (UID: \"369c0515-c0a1-4042-bcd9-8efbe2663bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.182503 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.182449    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/369c0515-c0a1-4042-bcd9-8efbe2663bd8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-nwb6h\" (UID: \"369c0515-c0a1-4042-bcd9-8efbe2663bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.283039 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283011    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-tls\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.283039 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283043    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.283276 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283065    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97787552-67e5-4f37-a9e9-8f53b7d66ca1-sys\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.283276 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283115    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-textfile\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.283276 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:48:28.283165    2572 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 25 06:48:28.283276 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283168    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/369c0515-c0a1-4042-bcd9-8efbe2663bd8-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-nwb6h\" (UID: \"369c0515-c0a1-4042-bcd9-8efbe2663bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.283276 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283202    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7hq4z\" (UniqueName: \"kubernetes.io/projected/97787552-67e5-4f37-a9e9-8f53b7d66ca1-kube-api-access-7hq4z\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.283276 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:48:28.283217    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-tls podName:97787552-67e5-4f37-a9e9-8f53b7d66ca1 nodeName:}" failed. No retries permitted until 2026-05-25 06:48:28.783199579 +0000 UTC m=+147.019385953 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-tls") pod "node-exporter-cwf8g" (UID: "97787552-67e5-4f37-a9e9-8f53b7d66ca1") : secret "node-exporter-tls" not found
May 25 06:48:28.283276 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283264    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zk76z\" (UniqueName: \"kubernetes.io/projected/369c0515-c0a1-4042-bcd9-8efbe2663bd8-kube-api-access-zk76z\") pod \"openshift-state-metrics-9c959559f-nwb6h\" (UID: \"369c0515-c0a1-4042-bcd9-8efbe2663bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.283674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283285    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/97787552-67e5-4f37-a9e9-8f53b7d66ca1-sys\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.283674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283298    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/369c0515-c0a1-4042-bcd9-8efbe2663bd8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-nwb6h\" (UID: \"369c0515-c0a1-4042-bcd9-8efbe2663bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.283674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283346    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97787552-67e5-4f37-a9e9-8f53b7d66ca1-metrics-client-ca\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.283674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283384    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-accelerators-collector-config\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.283674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283419    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-wtmp\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.283674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283467    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97787552-67e5-4f37-a9e9-8f53b7d66ca1-root\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.283674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283502    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/369c0515-c0a1-4042-bcd9-8efbe2663bd8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-nwb6h\" (UID: \"369c0515-c0a1-4042-bcd9-8efbe2663bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.283674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283546    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-textfile\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.283674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283588    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/97787552-67e5-4f37-a9e9-8f53b7d66ca1-root\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.283674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283639    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-wtmp\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.284134 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.283960    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/97787552-67e5-4f37-a9e9-8f53b7d66ca1-metrics-client-ca\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.284193 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.284166    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-accelerators-collector-config\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.284372 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.284352    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/369c0515-c0a1-4042-bcd9-8efbe2663bd8-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-nwb6h\" (UID: \"369c0515-c0a1-4042-bcd9-8efbe2663bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.285532 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.285505    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.285704 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.285682    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/369c0515-c0a1-4042-bcd9-8efbe2663bd8-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-nwb6h\" (UID: \"369c0515-c0a1-4042-bcd9-8efbe2663bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.285790 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.285775    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/369c0515-c0a1-4042-bcd9-8efbe2663bd8-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-nwb6h\" (UID: \"369c0515-c0a1-4042-bcd9-8efbe2663bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.290918 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.290894    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7hq4z\" (UniqueName: \"kubernetes.io/projected/97787552-67e5-4f37-a9e9-8f53b7d66ca1-kube-api-access-7hq4z\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.291675 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.291655    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zk76z\" (UniqueName: \"kubernetes.io/projected/369c0515-c0a1-4042-bcd9-8efbe2663bd8-kube-api-access-zk76z\") pod \"openshift-state-metrics-9c959559f-nwb6h\" (UID: \"369c0515-c0a1-4042-bcd9-8efbe2663bd8\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.425993 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.425896    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"
May 25 06:48:28.551693 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.551584    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h"]
May 25 06:48:28.554365 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:28.554337    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod369c0515_c0a1_4042_bcd9_8efbe2663bd8.slice/crio-470c75fba3a78ab9b6bcef1e5d63af5c4b78f77661b3cdab5947a6b3261cc0c9 WatchSource:0}: Error finding container 470c75fba3a78ab9b6bcef1e5d63af5c4b78f77661b3cdab5947a6b3261cc0c9: Status 404 returned error can't find the container with id 470c75fba3a78ab9b6bcef1e5d63af5c4b78f77661b3cdab5947a6b3261cc0c9
May 25 06:48:28.788137 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.788051    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-tls\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.790284 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.790267    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/97787552-67e5-4f37-a9e9-8f53b7d66ca1-node-exporter-tls\") pod \"node-exporter-cwf8g\" (UID: \"97787552-67e5-4f37-a9e9-8f53b7d66ca1\") " pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:28.848078 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.848045    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h" event={"ID":"369c0515-c0a1-4042-bcd9-8efbe2663bd8","Type":"ContainerStarted","Data":"cd3a517e76a1407b2ff2f68e8b8d7b97a585380e24d5bf1c16d0f81d85644150"}
May 25 06:48:28.848222 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.848083    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h" event={"ID":"369c0515-c0a1-4042-bcd9-8efbe2663bd8","Type":"ContainerStarted","Data":"e92b51ac8655fa09f717396a3b6e1f9d8f5b4e7b5f54e03f22147117fb3d779a"}
May 25 06:48:28.848222 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:28.848097    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h" event={"ID":"369c0515-c0a1-4042-bcd9-8efbe2663bd8","Type":"ContainerStarted","Data":"470c75fba3a78ab9b6bcef1e5d63af5c4b78f77661b3cdab5947a6b3261cc0c9"}
May 25 06:48:29.055111 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:29.055033    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-cwf8g"
May 25 06:48:29.063584 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:29.063553    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod97787552_67e5_4f37_a9e9_8f53b7d66ca1.slice/crio-b25fc797e3e335f1d0176349920b7764ff82b5b0f5149b1dbce545fe22d96a95 WatchSource:0}: Error finding container b25fc797e3e335f1d0176349920b7764ff82b5b0f5149b1dbce545fe22d96a95: Status 404 returned error can't find the container with id b25fc797e3e335f1d0176349920b7764ff82b5b0f5149b1dbce545fe22d96a95
May 25 06:48:29.852007 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:29.851938    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cwf8g" event={"ID":"97787552-67e5-4f37-a9e9-8f53b7d66ca1","Type":"ContainerStarted","Data":"b25fc797e3e335f1d0176349920b7764ff82b5b0f5149b1dbce545fe22d96a95"}
May 25 06:48:29.853957 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:29.853916    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h" event={"ID":"369c0515-c0a1-4042-bcd9-8efbe2663bd8","Type":"ContainerStarted","Data":"d7e5b2985835a52567869a9971de9ece1dd0c1c354490da955b238bd80bdcc34"}
May 25 06:48:29.871705 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:29.871656    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9c959559f-nwb6h" podStartSLOduration=0.752023244 podStartE2EDuration="1.871636334s" podCreationTimestamp="2026-05-25 06:48:28 +0000 UTC" firstStartedPulling="2026-05-25 06:48:28.657973539 +0000 UTC m=+146.894159912" lastFinishedPulling="2026-05-25 06:48:29.777586628 +0000 UTC m=+148.013773002" observedRunningTime="2026-05-25 06:48:29.869870159 +0000 UTC m=+148.106056547" watchObservedRunningTime="2026-05-25 06:48:29.871636334 +0000 UTC m=+148.107822940"
May 25 06:48:30.858218 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:30.858180    2572 generic.go:358] "Generic (PLEG): container finished" podID="97787552-67e5-4f37-a9e9-8f53b7d66ca1" containerID="16e92d08e507f92cb4ba490420b1c9297df755e8ef0502d1d2e6a76f6e02c37d" exitCode=0
May 25 06:48:30.858646 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:30.858268    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cwf8g" event={"ID":"97787552-67e5-4f37-a9e9-8f53b7d66ca1","Type":"ContainerDied","Data":"16e92d08e507f92cb4ba490420b1c9297df755e8ef0502d1d2e6a76f6e02c37d"}
May 25 06:48:31.862126 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:31.862087    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cwf8g" event={"ID":"97787552-67e5-4f37-a9e9-8f53b7d66ca1","Type":"ContainerStarted","Data":"da0350e9e3bb8e09190937c9feb9b70268bb8fa0928d2ef5dec89057e4b3c051"}
May 25 06:48:31.862126 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:31.862128    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-cwf8g" event={"ID":"97787552-67e5-4f37-a9e9-8f53b7d66ca1","Type":"ContainerStarted","Data":"4330e09391c0d87485779ec5626ede0661e01adc5094486e2a47013b9a2e0011"}
May 25 06:48:31.884737 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:31.884695    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-cwf8g" podStartSLOduration=2.874632177 podStartE2EDuration="3.88467943s" podCreationTimestamp="2026-05-25 06:48:28 +0000 UTC" firstStartedPulling="2026-05-25 06:48:29.065392546 +0000 UTC m=+147.301578916" lastFinishedPulling="2026-05-25 06:48:30.075439796 +0000 UTC m=+148.311626169" observedRunningTime="2026-05-25 06:48:31.88319516 +0000 UTC m=+150.119381552" watchObservedRunningTime="2026-05-25 06:48:31.88467943 +0000 UTC m=+150.120865822"
May 25 06:48:32.533167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.533141    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-5858b6cf8-nl6rr"]
May 25 06:48:32.536210 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.536187    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.538356 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.538331    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
May 25 06:48:32.539027 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.538966    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
May 25 06:48:32.539027 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.539022    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-jlih8ajbktln\""
May 25 06:48:32.539210 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.539038    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 25 06:48:32.539210 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.539039    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
May 25 06:48:32.539210 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.539041    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-zkgq8\""
May 25 06:48:32.549772 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.549752    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5858b6cf8-nl6rr"]
May 25 06:48:32.619039 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.619004    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-client-ca-bundle\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.619039 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.619041    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-secret-metrics-server-client-certs\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.619231 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.619077    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-metrics-server-audit-profiles\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.619231 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.619109    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-secret-metrics-server-tls\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.619231 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.619129    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-audit-log\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.619231 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.619181    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.619387 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.619231    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7lvmz\" (UniqueName: \"kubernetes.io/projected/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-kube-api-access-7lvmz\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.720232 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.720183    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-metrics-server-audit-profiles\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.720232 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.720246    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-secret-metrics-server-tls\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.720430 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.720265    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-audit-log\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.720430 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.720289    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.720430 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.720306    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7lvmz\" (UniqueName: \"kubernetes.io/projected/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-kube-api-access-7lvmz\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.720430 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.720352    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-client-ca-bundle\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.720430 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.720380    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-secret-metrics-server-client-certs\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.720771 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.720746    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-audit-log\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.721107 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.721074    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.721256 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.721239    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-metrics-server-audit-profiles\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.722873 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.722833    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-client-ca-bundle\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.723071 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.723049    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-secret-metrics-server-tls\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.723115 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.723049    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-secret-metrics-server-client-certs\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.728258 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.728234    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7lvmz\" (UniqueName: \"kubernetes.io/projected/40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0-kube-api-access-7lvmz\") pod \"metrics-server-5858b6cf8-nl6rr\" (UID: \"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0\") " pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.846002 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.845963    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:32.909076 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.908979    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7"]
May 25 06:48:32.913417 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.913395    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7"
May 25 06:48:32.915379 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.915362    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-28r4l\""
May 25 06:48:32.915492 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.915361    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
May 25 06:48:32.920411 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.920392    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7"]
May 25 06:48:32.975923 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:32.975899    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-5858b6cf8-nl6rr"]
May 25 06:48:32.978138 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:32.978117    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod40477ac9_6d9e_438c_a2ad_3f18c1dcb0a0.slice/crio-4f9963cf58fce7274ca0be7db2df440990b0e58200c1bb047068ffaa4ab444d8 WatchSource:0}: Error finding container 4f9963cf58fce7274ca0be7db2df440990b0e58200c1bb047068ffaa4ab444d8: Status 404 returned error can't find the container with id 4f9963cf58fce7274ca0be7db2df440990b0e58200c1bb047068ffaa4ab444d8
May 25 06:48:33.022485 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:33.022457    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7be6e3f-d7bd-4709-bc19-aef212c81644-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-fs5x7\" (UID: \"b7be6e3f-d7bd-4709-bc19-aef212c81644\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7"
May 25 06:48:33.123598 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:33.123506    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7be6e3f-d7bd-4709-bc19-aef212c81644-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-fs5x7\" (UID: \"b7be6e3f-d7bd-4709-bc19-aef212c81644\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7"
May 25 06:48:33.123753 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:48:33.123650    2572 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
May 25 06:48:33.123753 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:48:33.123724    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/b7be6e3f-d7bd-4709-bc19-aef212c81644-monitoring-plugin-cert podName:b7be6e3f-d7bd-4709-bc19-aef212c81644 nodeName:}" failed. No retries permitted until 2026-05-25 06:48:33.623708208 +0000 UTC m=+151.859894578 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/b7be6e3f-d7bd-4709-bc19-aef212c81644-monitoring-plugin-cert") pod "monitoring-plugin-665c9c8584-fs5x7" (UID: "b7be6e3f-d7bd-4709-bc19-aef212c81644") : secret "monitoring-plugin-cert" not found
May 25 06:48:33.628232 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:33.628197    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7be6e3f-d7bd-4709-bc19-aef212c81644-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-fs5x7\" (UID: \"b7be6e3f-d7bd-4709-bc19-aef212c81644\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7"
May 25 06:48:33.630577 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:33.630555    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/b7be6e3f-d7bd-4709-bc19-aef212c81644-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-fs5x7\" (UID: \"b7be6e3f-d7bd-4709-bc19-aef212c81644\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7"
May 25 06:48:33.823096 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:33.823054    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7"
May 25 06:48:33.872699 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:33.872661    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr" event={"ID":"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0","Type":"ContainerStarted","Data":"4f9963cf58fce7274ca0be7db2df440990b0e58200c1bb047068ffaa4ab444d8"}
May 25 06:48:33.941554 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:33.941523    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7"]
May 25 06:48:33.945890 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:33.945864    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb7be6e3f_d7bd_4709_bc19_aef212c81644.slice/crio-38dc3883bb1a8735880a2371e6c0a349d6cea1a7ac1ce76db098148bd3859d1b WatchSource:0}: Error finding container 38dc3883bb1a8735880a2371e6c0a349d6cea1a7ac1ce76db098148bd3859d1b: Status 404 returned error can't find the container with id 38dc3883bb1a8735880a2371e6c0a349d6cea1a7ac1ce76db098148bd3859d1b
May 25 06:48:34.069550 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:34.069509    2572 patch_prober.go:28] interesting pod/image-registry-78ddd5c598-9msrl container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 25 06:48:34.069710 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:34.069577    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl" podUID="815efb61-c2de-4c40-a2f5-9f4f9a4737b3" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 25 06:48:34.411267 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:34.411233    2572 patch_prober.go:28] interesting pod/image-registry-78494c4cc8-g2vcq container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 25 06:48:34.411458 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:34.411290    2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq" podUID="ebfad36b-e93a-41b2-8a88-94d1352d3f7d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 25 06:48:34.877092 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:34.877031    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr" event={"ID":"40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0","Type":"ContainerStarted","Data":"3568104aa8df6baee000a928d2ed28afaf05927ecff4059bcc89972b430b74d8"}
May 25 06:48:34.878657 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:34.878616    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7" event={"ID":"b7be6e3f-d7bd-4709-bc19-aef212c81644","Type":"ContainerStarted","Data":"38dc3883bb1a8735880a2371e6c0a349d6cea1a7ac1ce76db098148bd3859d1b"}
May 25 06:48:34.894147 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:34.894094    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr" podStartSLOduration=1.084063147 podStartE2EDuration="2.894079681s" podCreationTimestamp="2026-05-25 06:48:32 +0000 UTC" firstStartedPulling="2026-05-25 06:48:32.979970694 +0000 UTC m=+151.216157065" lastFinishedPulling="2026-05-25 06:48:34.789987224 +0000 UTC m=+153.026173599" observedRunningTime="2026-05-25 06:48:34.89301016 +0000 UTC m=+153.129196552" watchObservedRunningTime="2026-05-25 06:48:34.894079681 +0000 UTC m=+153.130266073"
May 25 06:48:35.808972 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:35.808883    2572 patch_prober.go:28] interesting pod/image-registry-78494c4cc8-g2vcq container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 25 06:48:35.808972 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:35.808940    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq" podUID="ebfad36b-e93a-41b2-8a88-94d1352d3f7d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 25 06:48:35.883858 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:35.883801    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7" event={"ID":"b7be6e3f-d7bd-4709-bc19-aef212c81644","Type":"ContainerStarted","Data":"5f4348a4ed1adc4431e9c6d827bb324a1a1bc28528ae4a6198e6e0c1fd7e82a2"}
May 25 06:48:35.884061 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:35.884017    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7"
May 25 06:48:35.888307 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:35.888287    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7"
May 25 06:48:35.897390 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:35.897348    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-665c9c8584-fs5x7" podStartSLOduration=2.318491489 podStartE2EDuration="3.897335973s" podCreationTimestamp="2026-05-25 06:48:32 +0000 UTC" firstStartedPulling="2026-05-25 06:48:33.94811473 +0000 UTC m=+152.184301106" lastFinishedPulling="2026-05-25 06:48:35.526950528 +0000 UTC m=+153.763145590" observedRunningTime="2026-05-25 06:48:35.896924712 +0000 UTC m=+154.133111103" watchObservedRunningTime="2026-05-25 06:48:35.897335973 +0000 UTC m=+154.133522403"
May 25 06:48:38.692340 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:48:38.692301    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-gz8v5" podUID="88e807ae-7d7e-426f-a3ab-25fa0d336a7f"
May 25 06:48:38.706906 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:48:38.706876    2572 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-x4v6p" podUID="f1ba578b-9195-4d91-be43-38f7a5820121"
May 25 06:48:38.895094 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:38.895066    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:48:38.895254 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:38.895073    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gz8v5"
May 25 06:48:39.083905 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.083835    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl" podUID="815efb61-c2de-4c40-a2f5-9f4f9a4737b3" containerName="registry" containerID="cri-o://fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8" gracePeriod=30
May 25 06:48:39.320601 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.320580    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:48:39.376457 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.376381    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls\") pod \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") "
May 25 06:48:39.376457 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.376422    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-installation-pull-secrets\") pod \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") "
May 25 06:48:39.376457 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.376456    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-trusted-ca\") pod \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") "
May 25 06:48:39.376690 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.376479    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ps28d\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-kube-api-access-ps28d\") pod \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") "
May 25 06:48:39.376690 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.376501    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-bound-sa-token\") pod \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") "
May 25 06:48:39.376690 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.376551    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-certificates\") pod \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") "
May 25 06:48:39.376690 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.376580    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-image-registry-private-configuration\") pod \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") "
May 25 06:48:39.376690 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.376613    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-ca-trust-extracted\") pod \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\" (UID: \"815efb61-c2de-4c40-a2f5-9f4f9a4737b3\") "
May 25 06:48:39.376983 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.376884    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "815efb61-c2de-4c40-a2f5-9f4f9a4737b3" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:48:39.377451 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.377416    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "815efb61-c2de-4c40-a2f5-9f4f9a4737b3" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:48:39.378899 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.378873    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-kube-api-access-ps28d" (OuterVolumeSpecName: "kube-api-access-ps28d") pod "815efb61-c2de-4c40-a2f5-9f4f9a4737b3" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3"). InnerVolumeSpecName "kube-api-access-ps28d". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 06:48:39.379004 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.378921    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "815efb61-c2de-4c40-a2f5-9f4f9a4737b3" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:48:39.379167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.379146    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "815efb61-c2de-4c40-a2f5-9f4f9a4737b3" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 06:48:39.379237 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.379154    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "815efb61-c2de-4c40-a2f5-9f4f9a4737b3" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:48:39.379237 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.379166    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "815efb61-c2de-4c40-a2f5-9f4f9a4737b3" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 06:48:39.385420 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.385400    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "815efb61-c2de-4c40-a2f5-9f4f9a4737b3" (UID: "815efb61-c2de-4c40-a2f5-9f4f9a4737b3"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 06:48:39.477407 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.477379    2572 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-installation-pull-secrets\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:48:39.477407 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.477405    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-trusted-ca\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:48:39.477407 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.477416    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ps28d\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-kube-api-access-ps28d\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:48:39.477407 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.477425    2572 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-bound-sa-token\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:48:39.477647 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.477434    2572 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-certificates\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:48:39.477647 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.477444    2572 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-image-registry-private-configuration\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:48:39.477647 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.477453    2572 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-ca-trust-extracted\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:48:39.477647 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.477462    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/815efb61-c2de-4c40-a2f5-9f4f9a4737b3-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:48:39.898371 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.898338    2572 generic.go:358] "Generic (PLEG): container finished" podID="815efb61-c2de-4c40-a2f5-9f4f9a4737b3" containerID="fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8" exitCode=0
May 25 06:48:39.898889 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.898404    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl"
May 25 06:48:39.898889 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.898423    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl" event={"ID":"815efb61-c2de-4c40-a2f5-9f4f9a4737b3","Type":"ContainerDied","Data":"fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8"}
May 25 06:48:39.898889 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.898462    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-78ddd5c598-9msrl" event={"ID":"815efb61-c2de-4c40-a2f5-9f4f9a4737b3","Type":"ContainerDied","Data":"7d140f33e86b296655ea9dc9ec690508c39d2762bdd77781b46dab53720532a7"}
May 25 06:48:39.898889 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.898488    2572 scope.go:117] "RemoveContainer" containerID="fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8"
May 25 06:48:39.906907 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.906890    2572 scope.go:117] "RemoveContainer" containerID="fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8"
May 25 06:48:39.907167 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:48:39.907140    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8\": container with ID starting with fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8 not found: ID does not exist" containerID="fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8"
May 25 06:48:39.907228 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.907175    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8"} err="failed to get container status \"fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8\": rpc error: code = NotFound desc = could not find container \"fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8\": container with ID starting with fa22c87f4da696c91d2994e48ac3b9fc640486e9c5e6305c4904efec2b2600d8 not found: ID does not exist"
May 25 06:48:39.917366 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.917341    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-78ddd5c598-9msrl"]
May 25 06:48:39.920778 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:39.920756    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-78ddd5c598-9msrl"]
May 25 06:48:40.282602 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:40.282516    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="815efb61-c2de-4c40-a2f5-9f4f9a4737b3" path="/var/lib/kubelet/pods/815efb61-c2de-4c40-a2f5-9f4f9a4737b3/volumes"
May 25 06:48:42.420032 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.419998    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-59f84bd494-c4nht"]
May 25 06:48:42.420443 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.420257    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="815efb61-c2de-4c40-a2f5-9f4f9a4737b3" containerName="registry"
May 25 06:48:42.420443 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.420267    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="815efb61-c2de-4c40-a2f5-9f4f9a4737b3" containerName="registry"
May 25 06:48:42.420443 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.420324    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="815efb61-c2de-4c40-a2f5-9f4f9a4737b3" containerName="registry"
May 25 06:48:42.425059 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.425040    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.430983 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.430569    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 25 06:48:42.430983 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.430580    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 25 06:48:42.430983 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.430826    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 25 06:48:42.430983 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.430915    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 25 06:48:42.431208 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.431102    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 25 06:48:42.431208 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.431157    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-v5r96\""
May 25 06:48:42.431662 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.431465    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 25 06:48:42.431662 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.431512    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 25 06:48:42.432511 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.432488    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59f84bd494-c4nht"]
May 25 06:48:42.503867 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.503808    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-serving-cert\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.504012 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.503910    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-djbgv\" (UniqueName: \"kubernetes.io/projected/cde4bdd3-07f0-4647-81b6-b7158e3c9247-kube-api-access-djbgv\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.504012 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.503949    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-oauth-config\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.504012 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.503977    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-config\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.504012 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.503997    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-oauth-serving-cert\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.504210 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.504156    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-service-ca\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.604680 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.604642    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-service-ca\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.604828 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.604696    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-serving-cert\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.604828 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.604726    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-djbgv\" (UniqueName: \"kubernetes.io/projected/cde4bdd3-07f0-4647-81b6-b7158e3c9247-kube-api-access-djbgv\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.604828 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.604757    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-oauth-config\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.604828 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.604790    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-config\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.604828 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.604816    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-oauth-serving-cert\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.605456 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.605427    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-service-ca\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.605565 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.605509    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-config\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.605630 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.605607    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-oauth-serving-cert\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.607201 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.607175    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-oauth-config\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.607342 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.607322    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-serving-cert\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.613584 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.613562    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-djbgv\" (UniqueName: \"kubernetes.io/projected/cde4bdd3-07f0-4647-81b6-b7158e3c9247-kube-api-access-djbgv\") pod \"console-59f84bd494-c4nht\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") " pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.738910 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.738797    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:42.870383 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.870353    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-59f84bd494-c4nht"]
May 25 06:48:42.873698 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:42.873674    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcde4bdd3_07f0_4647_81b6_b7158e3c9247.slice/crio-7892909ee991569be8474f851467ae952d1752f188a6fffb9ff7cb7851e2c945 WatchSource:0}: Error finding container 7892909ee991569be8474f851467ae952d1752f188a6fffb9ff7cb7851e2c945: Status 404 returned error can't find the container with id 7892909ee991569be8474f851467ae952d1752f188a6fffb9ff7cb7851e2c945
May 25 06:48:42.908452 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:42.908417    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59f84bd494-c4nht" event={"ID":"cde4bdd3-07f0-4647-81b6-b7158e3c9247","Type":"ContainerStarted","Data":"7892909ee991569be8474f851467ae952d1752f188a6fffb9ff7cb7851e2c945"}
May 25 06:48:43.613800 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:43.613767    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:48:43.614271 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:43.613816    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:48:43.616541 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:43.616513    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/88e807ae-7d7e-426f-a3ab-25fa0d336a7f-metrics-tls\") pod \"dns-default-gz8v5\" (UID: \"88e807ae-7d7e-426f-a3ab-25fa0d336a7f\") " pod="openshift-dns/dns-default-gz8v5"
May 25 06:48:43.616667 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:43.616577    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/f1ba578b-9195-4d91-be43-38f7a5820121-cert\") pod \"ingress-canary-x4v6p\" (UID: \"f1ba578b-9195-4d91-be43-38f7a5820121\") " pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:48:43.698197 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:43.698166    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-m6t2j\""
May 25 06:48:43.698368 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:43.698277    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-5nl6q\""
May 25 06:48:43.706761 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:43.706729    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-x4v6p"
May 25 06:48:43.706951 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:43.706829    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-gz8v5"
May 25 06:48:43.861598 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:43.861387    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-gz8v5"]
May 25 06:48:43.864475 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:43.864402    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod88e807ae_7d7e_426f_a3ab_25fa0d336a7f.slice/crio-2fc15659b345f27aca6318a5bd913e11dba7a64697a8718abd075fc3b1e9abda WatchSource:0}: Error finding container 2fc15659b345f27aca6318a5bd913e11dba7a64697a8718abd075fc3b1e9abda: Status 404 returned error can't find the container with id 2fc15659b345f27aca6318a5bd913e11dba7a64697a8718abd075fc3b1e9abda
May 25 06:48:43.878914 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:43.878779    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-x4v6p"]
May 25 06:48:43.881589 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:43.881560    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf1ba578b_9195_4d91_be43_38f7a5820121.slice/crio-c1e8d0a0f6350973013b023ce1689fa34c0f053c65f8425ab1b9b6c59b30acd3 WatchSource:0}: Error finding container c1e8d0a0f6350973013b023ce1689fa34c0f053c65f8425ab1b9b6c59b30acd3: Status 404 returned error can't find the container with id c1e8d0a0f6350973013b023ce1689fa34c0f053c65f8425ab1b9b6c59b30acd3
May 25 06:48:43.912413 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:43.912380    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gz8v5" event={"ID":"88e807ae-7d7e-426f-a3ab-25fa0d336a7f","Type":"ContainerStarted","Data":"2fc15659b345f27aca6318a5bd913e11dba7a64697a8718abd075fc3b1e9abda"}
May 25 06:48:43.913475 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:43.913433    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x4v6p" event={"ID":"f1ba578b-9195-4d91-be43-38f7a5820121","Type":"ContainerStarted","Data":"c1e8d0a0f6350973013b023ce1689fa34c0f053c65f8425ab1b9b6c59b30acd3"}
May 25 06:48:44.413253 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:44.413214    2572 patch_prober.go:28] interesting pod/image-registry-78494c4cc8-g2vcq container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 25 06:48:44.413424 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:44.413278    2572 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq" podUID="ebfad36b-e93a-41b2-8a88-94d1352d3f7d" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 25 06:48:45.808824 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:45.808793    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-78494c4cc8-g2vcq"
May 25 06:48:46.923260 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:46.923170    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gz8v5" event={"ID":"88e807ae-7d7e-426f-a3ab-25fa0d336a7f","Type":"ContainerStarted","Data":"36a57bf136fdabcbe9a5270de872f24fd380e237194f7239e191c0cc40f59520"}
May 25 06:48:46.923260 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:46.923207    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-gz8v5" event={"ID":"88e807ae-7d7e-426f-a3ab-25fa0d336a7f","Type":"ContainerStarted","Data":"39b2a435c5502e39681d534d03c6a6d8be5356e2d01d6db7de0b710481f22cf7"}
May 25 06:48:46.923720 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:46.923266    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-gz8v5"
May 25 06:48:46.924572 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:46.924552    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59f84bd494-c4nht" event={"ID":"cde4bdd3-07f0-4647-81b6-b7158e3c9247","Type":"ContainerStarted","Data":"463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709"}
May 25 06:48:46.925731 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:46.925711    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-x4v6p" event={"ID":"f1ba578b-9195-4d91-be43-38f7a5820121","Type":"ContainerStarted","Data":"de95ba08d42c3a40e0c695471bb797075189183bdf6ee7e40b2ee19c45deff1f"}
May 25 06:48:46.939917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:46.939878    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-gz8v5" podStartSLOduration=129.160312578 podStartE2EDuration="2m11.939864571s" podCreationTimestamp="2026-05-25 06:46:35 +0000 UTC" firstStartedPulling="2026-05-25 06:48:43.866833518 +0000 UTC m=+162.103019894" lastFinishedPulling="2026-05-25 06:48:46.646385503 +0000 UTC m=+164.882571887" observedRunningTime="2026-05-25 06:48:46.938198301 +0000 UTC m=+165.174384693" watchObservedRunningTime="2026-05-25 06:48:46.939864571 +0000 UTC m=+165.176050962"
May 25 06:48:46.956695 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:46.956656    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-x4v6p" podStartSLOduration=129.191986997 podStartE2EDuration="2m11.956639397s" podCreationTimestamp="2026-05-25 06:46:35 +0000 UTC" firstStartedPulling="2026-05-25 06:48:43.884996026 +0000 UTC m=+162.121182416" lastFinishedPulling="2026-05-25 06:48:46.649648446 +0000 UTC m=+164.885834816" observedRunningTime="2026-05-25 06:48:46.955835914 +0000 UTC m=+165.192022305" watchObservedRunningTime="2026-05-25 06:48:46.956639397 +0000 UTC m=+165.192825791"
May 25 06:48:46.972516 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:46.972298    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-59f84bd494-c4nht" podStartSLOduration=1.202594906 podStartE2EDuration="4.972279055s" podCreationTimestamp="2026-05-25 06:48:42 +0000 UTC" firstStartedPulling="2026-05-25 06:48:42.87555845 +0000 UTC m=+161.111744823" lastFinishedPulling="2026-05-25 06:48:46.645242596 +0000 UTC m=+164.881428972" observedRunningTime="2026-05-25 06:48:46.971151544 +0000 UTC m=+165.207337936" watchObservedRunningTime="2026-05-25 06:48:46.972279055 +0000 UTC m=+165.208465449"
May 25 06:48:51.804604 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.804526    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-8bf7d7d4-zkxdd"]
May 25 06:48:51.811542 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.811519    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.817900 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.817815    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8bf7d7d4-zkxdd"]
May 25 06:48:51.819830 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.819803    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
May 25 06:48:51.877481 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.877447    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz2xm\" (UniqueName: \"kubernetes.io/projected/a5654d3d-84b5-436f-8dbb-05de2dd05478-kube-api-access-dz2xm\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.877644 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.877490    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-serving-cert\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.877644 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.877513    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-service-ca\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.877644 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.877553    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-config\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.877644 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.877578    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-oauth-config\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.877644 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.877599    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-oauth-serving-cert\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.877802 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.877650    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-trusted-ca-bundle\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.978387 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.978355    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-service-ca\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.978560 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.978397    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-config\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.978560 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.978423    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-oauth-config\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.978560 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.978452    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-oauth-serving-cert\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.978560 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.978469    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-trusted-ca-bundle\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.978560 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.978517    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dz2xm\" (UniqueName: \"kubernetes.io/projected/a5654d3d-84b5-436f-8dbb-05de2dd05478-kube-api-access-dz2xm\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.978560 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.978560    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-serving-cert\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.979261 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.979230    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-service-ca\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.979354 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.979269    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-oauth-serving-cert\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.979354 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.979346    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-config\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.979454 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.979432    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-trusted-ca-bundle\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.981333 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.981304    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-oauth-config\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.981441 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.981407    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-serving-cert\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:51.988455 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:51.988432    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz2xm\" (UniqueName: \"kubernetes.io/projected/a5654d3d-84b5-436f-8dbb-05de2dd05478-kube-api-access-dz2xm\") pod \"console-8bf7d7d4-zkxdd\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") " pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:52.121138 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:52.121103    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:48:52.289604 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:52.289576    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-8bf7d7d4-zkxdd"]
May 25 06:48:52.292389 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:48:52.292363    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda5654d3d_84b5_436f_8dbb_05de2dd05478.slice/crio-8732f445f70eb64aa10b36a4efa3be7ed6ec9c53e14f64014775654afec6cfd3 WatchSource:0}: Error finding container 8732f445f70eb64aa10b36a4efa3be7ed6ec9c53e14f64014775654afec6cfd3: Status 404 returned error can't find the container with id 8732f445f70eb64aa10b36a4efa3be7ed6ec9c53e14f64014775654afec6cfd3
May 25 06:48:52.739650 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:52.739616    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:52.739804 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:52.739695    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:52.744311 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:52.744291    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:52.846263 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:52.846222    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:52.846632 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:52.846301    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:48:52.942435 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:52.942400    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bf7d7d4-zkxdd" event={"ID":"a5654d3d-84b5-436f-8dbb-05de2dd05478","Type":"ContainerStarted","Data":"d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315"}
May 25 06:48:52.942435 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:52.942436    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bf7d7d4-zkxdd" event={"ID":"a5654d3d-84b5-436f-8dbb-05de2dd05478","Type":"ContainerStarted","Data":"8732f445f70eb64aa10b36a4efa3be7ed6ec9c53e14f64014775654afec6cfd3"}
May 25 06:48:52.946801 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:52.946777    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:48:52.969242 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:52.969195    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-8bf7d7d4-zkxdd" podStartSLOduration=1.9691804130000001 podStartE2EDuration="1.969180413s" podCreationTimestamp="2026-05-25 06:48:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 06:48:52.966508155 +0000 UTC m=+171.202694547" watchObservedRunningTime="2026-05-25 06:48:52.969180413 +0000 UTC m=+171.205366805"
May 25 06:48:56.930256 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:48:56.930225    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-gz8v5"
May 25 06:49:00.968157 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:00.968121    2572 generic.go:358] "Generic (PLEG): container finished" podID="8d2c6188-c2a6-43cd-a2ec-f7eb401bb061" containerID="730af3ceb93cbf0f222ecafd5ffe68825c1ff5f6571fe2ce93075a6f8e27bb1e" exitCode=0
May 25 06:49:00.968612 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:00.968200    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4" event={"ID":"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061","Type":"ContainerDied","Data":"730af3ceb93cbf0f222ecafd5ffe68825c1ff5f6571fe2ce93075a6f8e27bb1e"}
May 25 06:49:00.968795 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:00.968774    2572 scope.go:117] "RemoveContainer" containerID="730af3ceb93cbf0f222ecafd5ffe68825c1ff5f6571fe2ce93075a6f8e27bb1e"
May 25 06:49:01.972706 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:01.972674    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-h9nz4" event={"ID":"8d2c6188-c2a6-43cd-a2ec-f7eb401bb061","Type":"ContainerStarted","Data":"a419503cefa8596ec1f815833eb86278e2ca3fd63d3d957830a40e9cc21193bd"}
May 25 06:49:02.121927 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:02.121890    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:49:02.122111 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:02.121948    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:49:02.126656 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:02.126633    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:49:02.979370 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:02.979342    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:49:03.025672 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:03.025634    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59f84bd494-c4nht"]
May 25 06:49:10.999136 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:10.999103    2572 generic.go:358] "Generic (PLEG): container finished" podID="a8af8432-7b6d-4e9c-acb6-fedaa77d523f" containerID="36570cec4479f32d6e7a0dcd892e52a12b7b552aecbf968cefc892f7d97cf8ca" exitCode=0
May 25 06:49:10.999497 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:10.999176    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74" event={"ID":"a8af8432-7b6d-4e9c-acb6-fedaa77d523f","Type":"ContainerDied","Data":"36570cec4479f32d6e7a0dcd892e52a12b7b552aecbf968cefc892f7d97cf8ca"}
May 25 06:49:10.999497 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:10.999489    2572 scope.go:117] "RemoveContainer" containerID="36570cec4479f32d6e7a0dcd892e52a12b7b552aecbf968cefc892f7d97cf8ca"
May 25 06:49:12.003639 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:12.003608    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-7vk74" event={"ID":"a8af8432-7b6d-4e9c-acb6-fedaa77d523f","Type":"ContainerStarted","Data":"3e5f8163f01b2af0738e09037d1a93d5a8c35e4885d0fa57e234a0761c188145"}
May 25 06:49:12.851747 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:12.851720    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:49:12.855550 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:12.855524    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-5858b6cf8-nl6rr"
May 25 06:49:28.044564 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.044523    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-59f84bd494-c4nht" podUID="cde4bdd3-07f0-4647-81b6-b7158e3c9247" containerName="console" containerID="cri-o://463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709" gracePeriod=15
May 25 06:49:28.281195 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.281171    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-59f84bd494-c4nht_cde4bdd3-07f0-4647-81b6-b7158e3c9247/console/0.log"
May 25 06:49:28.281340 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.281234    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:49:28.371270 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.371233    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-oauth-serving-cert\") pod \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") "
May 25 06:49:28.371439 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.371278    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-djbgv\" (UniqueName: \"kubernetes.io/projected/cde4bdd3-07f0-4647-81b6-b7158e3c9247-kube-api-access-djbgv\") pod \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") "
May 25 06:49:28.371439 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.371320    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-serving-cert\") pod \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") "
May 25 06:49:28.371439 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.371375    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-config\") pod \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") "
May 25 06:49:28.371695 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.371673    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "cde4bdd3-07f0-4647-81b6-b7158e3c9247" (UID: "cde4bdd3-07f0-4647-81b6-b7158e3c9247"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:49:28.371892 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.371832    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-config" (OuterVolumeSpecName: "console-config") pod "cde4bdd3-07f0-4647-81b6-b7158e3c9247" (UID: "cde4bdd3-07f0-4647-81b6-b7158e3c9247"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:49:28.373614 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.373584    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "cde4bdd3-07f0-4647-81b6-b7158e3c9247" (UID: "cde4bdd3-07f0-4647-81b6-b7158e3c9247"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:49:28.373614 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.373595    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cde4bdd3-07f0-4647-81b6-b7158e3c9247-kube-api-access-djbgv" (OuterVolumeSpecName: "kube-api-access-djbgv") pod "cde4bdd3-07f0-4647-81b6-b7158e3c9247" (UID: "cde4bdd3-07f0-4647-81b6-b7158e3c9247"). InnerVolumeSpecName "kube-api-access-djbgv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 06:49:28.472085 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.472053    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-oauth-config\") pod \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") "
May 25 06:49:28.472259 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.472093    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-service-ca\") pod \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\" (UID: \"cde4bdd3-07f0-4647-81b6-b7158e3c9247\") "
May 25 06:49:28.472259 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.472216    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-oauth-serving-cert\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:49:28.472259 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.472228    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-djbgv\" (UniqueName: \"kubernetes.io/projected/cde4bdd3-07f0-4647-81b6-b7158e3c9247-kube-api-access-djbgv\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:49:28.472259 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.472239    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-serving-cert\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:49:28.472259 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.472248    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-config\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:49:28.472540 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.472512    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-service-ca" (OuterVolumeSpecName: "service-ca") pod "cde4bdd3-07f0-4647-81b6-b7158e3c9247" (UID: "cde4bdd3-07f0-4647-81b6-b7158e3c9247"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:49:28.474072 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.474052    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "cde4bdd3-07f0-4647-81b6-b7158e3c9247" (UID: "cde4bdd3-07f0-4647-81b6-b7158e3c9247"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:49:28.573144 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.573117    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cde4bdd3-07f0-4647-81b6-b7158e3c9247-console-oauth-config\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:49:28.573144 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:28.573143    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cde4bdd3-07f0-4647-81b6-b7158e3c9247-service-ca\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:49:29.050672 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:29.050646    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-59f84bd494-c4nht_cde4bdd3-07f0-4647-81b6-b7158e3c9247/console/0.log"
May 25 06:49:29.051071 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:29.050689    2572 generic.go:358] "Generic (PLEG): container finished" podID="cde4bdd3-07f0-4647-81b6-b7158e3c9247" containerID="463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709" exitCode=2
May 25 06:49:29.051071 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:29.050737    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59f84bd494-c4nht" event={"ID":"cde4bdd3-07f0-4647-81b6-b7158e3c9247","Type":"ContainerDied","Data":"463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709"}
May 25 06:49:29.051071 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:29.050765    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-59f84bd494-c4nht"
May 25 06:49:29.051071 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:29.050780    2572 scope.go:117] "RemoveContainer" containerID="463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709"
May 25 06:49:29.051071 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:29.050767    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-59f84bd494-c4nht" event={"ID":"cde4bdd3-07f0-4647-81b6-b7158e3c9247","Type":"ContainerDied","Data":"7892909ee991569be8474f851467ae952d1752f188a6fffb9ff7cb7851e2c945"}
May 25 06:49:29.060066 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:29.060049    2572 scope.go:117] "RemoveContainer" containerID="463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709"
May 25 06:49:29.060331 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:49:29.060309    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709\": container with ID starting with 463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709 not found: ID does not exist" containerID="463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709"
May 25 06:49:29.060418 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:29.060339    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709"} err="failed to get container status \"463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709\": rpc error: code = NotFound desc = could not find container \"463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709\": container with ID starting with 463e7711f664d432324a08ac27a42a885a2b81e8fcdcd44482e15c64c6a6c709 not found: ID does not exist"
May 25 06:49:29.070980 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:29.070956    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-59f84bd494-c4nht"]
May 25 06:49:29.079074 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:29.079049    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-59f84bd494-c4nht"]
May 25 06:49:30.281722 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:49:30.281692    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cde4bdd3-07f0-4647-81b6-b7158e3c9247" path="/var/lib/kubelet/pods/cde4bdd3-07f0-4647-81b6-b7158e3c9247/volumes"
May 25 06:50:01.285108 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.285071    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-687cf575c8-ps6n6"]
May 25 06:50:01.285700 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.285359    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cde4bdd3-07f0-4647-81b6-b7158e3c9247" containerName="console"
May 25 06:50:01.285700 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.285369    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cde4bdd3-07f0-4647-81b6-b7158e3c9247" containerName="console"
May 25 06:50:01.285700 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.285430    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="cde4bdd3-07f0-4647-81b6-b7158e3c9247" containerName="console"
May 25 06:50:01.288350 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.288334    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.297276 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.297253    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-687cf575c8-ps6n6"]
May 25 06:50:01.301409 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.301386    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-serving-cert\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.301587 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.301419    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-oauth-serving-cert\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.301587 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.301442    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9xq6t\" (UniqueName: \"kubernetes.io/projected/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-kube-api-access-9xq6t\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.301587 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.301489    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-config\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.301587 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.301534    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-trusted-ca-bundle\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.301752 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.301590    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-oauth-config\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.301752 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.301612    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-service-ca\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.402149 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.402107    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-oauth-config\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.402346 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.402150    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-service-ca\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.402346 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.402256    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-serving-cert\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.402346 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.402290    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-oauth-serving-cert\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.402346 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.402321    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9xq6t\" (UniqueName: \"kubernetes.io/projected/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-kube-api-access-9xq6t\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.402564 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.402474    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-config\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.402564 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.402536    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-trusted-ca-bundle\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.403091 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.403066    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-oauth-serving-cert\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.403223 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.403065    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-service-ca\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.403223 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.403160    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-config\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.403706 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.403683    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-trusted-ca-bundle\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.404814 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.404795    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-serving-cert\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.404878 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.404816    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-oauth-config\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.409783 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.409760    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9xq6t\" (UniqueName: \"kubernetes.io/projected/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-kube-api-access-9xq6t\") pod \"console-687cf575c8-ps6n6\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") " pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.598974 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.598926    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:01.729991 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:01.729968    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-687cf575c8-ps6n6"]
May 25 06:50:01.732645 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:50:01.732616    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc3ba1d94_4e5b_4f2d_bc27_a2a079617a83.slice/crio-1a93be3fce2b1d31d438c5abaedb91efb54b3e07f0e371c6320ed3e2a88078df WatchSource:0}: Error finding container 1a93be3fce2b1d31d438c5abaedb91efb54b3e07f0e371c6320ed3e2a88078df: Status 404 returned error can't find the container with id 1a93be3fce2b1d31d438c5abaedb91efb54b3e07f0e371c6320ed3e2a88078df
May 25 06:50:02.146320 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:02.146275    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-687cf575c8-ps6n6" event={"ID":"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83","Type":"ContainerStarted","Data":"954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee"}
May 25 06:50:02.146320 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:02.146317    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-687cf575c8-ps6n6" event={"ID":"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83","Type":"ContainerStarted","Data":"1a93be3fce2b1d31d438c5abaedb91efb54b3e07f0e371c6320ed3e2a88078df"}
May 25 06:50:02.163570 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:02.163448    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-687cf575c8-ps6n6" podStartSLOduration=1.163429368 podStartE2EDuration="1.163429368s" podCreationTimestamp="2026-05-25 06:50:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 06:50:02.162450151 +0000 UTC m=+240.398636544" watchObservedRunningTime="2026-05-25 06:50:02.163429368 +0000 UTC m=+240.399615762"
May 25 06:50:11.599090 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:11.599049    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:11.599568 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:11.599100    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:11.603909 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:11.603887    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:12.180261 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:12.180234    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:50:12.225274 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:12.225240    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8bf7d7d4-zkxdd"]
May 25 06:50:23.564541 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.564456    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-6rmxj"]
May 25 06:50:23.567672 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.567651    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6rmxj"
May 25 06:50:23.570809 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.570781    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 25 06:50:23.575173 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.575149    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6rmxj"]
May 25 06:50:23.677269 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.677233    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418-dbus\") pod \"global-pull-secret-syncer-6rmxj\" (UID: \"0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418\") " pod="kube-system/global-pull-secret-syncer-6rmxj"
May 25 06:50:23.677477 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.677335    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418-original-pull-secret\") pod \"global-pull-secret-syncer-6rmxj\" (UID: \"0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418\") " pod="kube-system/global-pull-secret-syncer-6rmxj"
May 25 06:50:23.677477 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.677377    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418-kubelet-config\") pod \"global-pull-secret-syncer-6rmxj\" (UID: \"0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418\") " pod="kube-system/global-pull-secret-syncer-6rmxj"
May 25 06:50:23.777795 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.777756    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418-original-pull-secret\") pod \"global-pull-secret-syncer-6rmxj\" (UID: \"0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418\") " pod="kube-system/global-pull-secret-syncer-6rmxj"
May 25 06:50:23.777795 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.777798    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418-kubelet-config\") pod \"global-pull-secret-syncer-6rmxj\" (UID: \"0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418\") " pod="kube-system/global-pull-secret-syncer-6rmxj"
May 25 06:50:23.778037 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.777832    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418-dbus\") pod \"global-pull-secret-syncer-6rmxj\" (UID: \"0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418\") " pod="kube-system/global-pull-secret-syncer-6rmxj"
May 25 06:50:23.778037 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.777946    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418-kubelet-config\") pod \"global-pull-secret-syncer-6rmxj\" (UID: \"0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418\") " pod="kube-system/global-pull-secret-syncer-6rmxj"
May 25 06:50:23.778037 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.777993    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418-dbus\") pod \"global-pull-secret-syncer-6rmxj\" (UID: \"0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418\") " pod="kube-system/global-pull-secret-syncer-6rmxj"
May 25 06:50:23.780037 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.780017    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418-original-pull-secret\") pod \"global-pull-secret-syncer-6rmxj\" (UID: \"0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418\") " pod="kube-system/global-pull-secret-syncer-6rmxj"
May 25 06:50:23.878224 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:23.878187    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-6rmxj"
May 25 06:50:24.005221 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:24.005182    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-6rmxj"]
May 25 06:50:24.008578 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:50:24.008543    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0406bbb3_eaa6_43dc_aef7_9b2b2b1a8418.slice/crio-583b78a245fe0bd847fef113b2c5ea41dd34547c7e377848c7d4707fa213acec WatchSource:0}: Error finding container 583b78a245fe0bd847fef113b2c5ea41dd34547c7e377848c7d4707fa213acec: Status 404 returned error can't find the container with id 583b78a245fe0bd847fef113b2c5ea41dd34547c7e377848c7d4707fa213acec
May 25 06:50:24.209044 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:24.208956    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6rmxj" event={"ID":"0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418","Type":"ContainerStarted","Data":"583b78a245fe0bd847fef113b2c5ea41dd34547c7e377848c7d4707fa213acec"}
May 25 06:50:28.221453 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:28.221416    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-6rmxj" event={"ID":"0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418","Type":"ContainerStarted","Data":"53817b03e08dcb71b930a59c6e3e5deb0c4e8c66aede6f8836e086042dc12171"}
May 25 06:50:28.235590 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:28.235532    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-6rmxj" podStartSLOduration=1.442395624 podStartE2EDuration="5.235514473s" podCreationTimestamp="2026-05-25 06:50:23 +0000 UTC" firstStartedPulling="2026-05-25 06:50:24.010121131 +0000 UTC m=+262.246307501" lastFinishedPulling="2026-05-25 06:50:27.803239959 +0000 UTC m=+266.039426350" observedRunningTime="2026-05-25 06:50:28.234819595 +0000 UTC m=+266.471005987" watchObservedRunningTime="2026-05-25 06:50:28.235514473 +0000 UTC m=+266.471700867"
May 25 06:50:37.248992 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.248935    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-8bf7d7d4-zkxdd" podUID="a5654d3d-84b5-436f-8dbb-05de2dd05478" containerName="console" containerID="cri-o://d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315" gracePeriod=15
May 25 06:50:37.496450 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.496425    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8bf7d7d4-zkxdd_a5654d3d-84b5-436f-8dbb-05de2dd05478/console/0.log"
May 25 06:50:37.496585 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.496493    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:50:37.591355 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.591322    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-serving-cert\") pod \"a5654d3d-84b5-436f-8dbb-05de2dd05478\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") "
May 25 06:50:37.591520 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.591401    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-config\") pod \"a5654d3d-84b5-436f-8dbb-05de2dd05478\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") "
May 25 06:50:37.591520 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.591425    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-oauth-serving-cert\") pod \"a5654d3d-84b5-436f-8dbb-05de2dd05478\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") "
May 25 06:50:37.591520 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.591454    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-oauth-config\") pod \"a5654d3d-84b5-436f-8dbb-05de2dd05478\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") "
May 25 06:50:37.591520 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.591503    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dz2xm\" (UniqueName: \"kubernetes.io/projected/a5654d3d-84b5-436f-8dbb-05de2dd05478-kube-api-access-dz2xm\") pod \"a5654d3d-84b5-436f-8dbb-05de2dd05478\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") "
May 25 06:50:37.591702 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.591527    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-service-ca\") pod \"a5654d3d-84b5-436f-8dbb-05de2dd05478\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") "
May 25 06:50:37.591702 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.591554    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-trusted-ca-bundle\") pod \"a5654d3d-84b5-436f-8dbb-05de2dd05478\" (UID: \"a5654d3d-84b5-436f-8dbb-05de2dd05478\") "
May 25 06:50:37.591977 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.591898    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "a5654d3d-84b5-436f-8dbb-05de2dd05478" (UID: "a5654d3d-84b5-436f-8dbb-05de2dd05478"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:50:37.592131 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.591908    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-config" (OuterVolumeSpecName: "console-config") pod "a5654d3d-84b5-436f-8dbb-05de2dd05478" (UID: "a5654d3d-84b5-436f-8dbb-05de2dd05478"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:50:37.592131 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.592013    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-service-ca" (OuterVolumeSpecName: "service-ca") pod "a5654d3d-84b5-436f-8dbb-05de2dd05478" (UID: "a5654d3d-84b5-436f-8dbb-05de2dd05478"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:50:37.592131 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.592088    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "a5654d3d-84b5-436f-8dbb-05de2dd05478" (UID: "a5654d3d-84b5-436f-8dbb-05de2dd05478"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:50:37.593696 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.593670    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "a5654d3d-84b5-436f-8dbb-05de2dd05478" (UID: "a5654d3d-84b5-436f-8dbb-05de2dd05478"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:50:37.593803 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.593719    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a5654d3d-84b5-436f-8dbb-05de2dd05478-kube-api-access-dz2xm" (OuterVolumeSpecName: "kube-api-access-dz2xm") pod "a5654d3d-84b5-436f-8dbb-05de2dd05478" (UID: "a5654d3d-84b5-436f-8dbb-05de2dd05478"). InnerVolumeSpecName "kube-api-access-dz2xm". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 06:50:37.593803 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.593730    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "a5654d3d-84b5-436f-8dbb-05de2dd05478" (UID: "a5654d3d-84b5-436f-8dbb-05de2dd05478"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:50:37.692541 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.692487    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dz2xm\" (UniqueName: \"kubernetes.io/projected/a5654d3d-84b5-436f-8dbb-05de2dd05478-kube-api-access-dz2xm\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:50:37.692541 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.692538    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-service-ca\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:50:37.692753 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.692561    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-trusted-ca-bundle\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:50:37.692753 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.692570    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-serving-cert\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:50:37.692753 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.692579    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-config\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:50:37.692753 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.692588    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/a5654d3d-84b5-436f-8dbb-05de2dd05478-oauth-serving-cert\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:50:37.692753 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:37.692596    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/a5654d3d-84b5-436f-8dbb-05de2dd05478-console-oauth-config\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:50:38.249439 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:38.249407    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-8bf7d7d4-zkxdd_a5654d3d-84b5-436f-8dbb-05de2dd05478/console/0.log"
May 25 06:50:38.249885 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:38.249447    2572 generic.go:358] "Generic (PLEG): container finished" podID="a5654d3d-84b5-436f-8dbb-05de2dd05478" containerID="d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315" exitCode=2
May 25 06:50:38.249885 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:38.249479    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bf7d7d4-zkxdd" event={"ID":"a5654d3d-84b5-436f-8dbb-05de2dd05478","Type":"ContainerDied","Data":"d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315"}
May 25 06:50:38.249885 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:38.249514    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-8bf7d7d4-zkxdd"
May 25 06:50:38.249885 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:38.249520    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-8bf7d7d4-zkxdd" event={"ID":"a5654d3d-84b5-436f-8dbb-05de2dd05478","Type":"ContainerDied","Data":"8732f445f70eb64aa10b36a4efa3be7ed6ec9c53e14f64014775654afec6cfd3"}
May 25 06:50:38.249885 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:38.249535    2572 scope.go:117] "RemoveContainer" containerID="d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315"
May 25 06:50:38.259132 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:38.259117    2572 scope.go:117] "RemoveContainer" containerID="d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315"
May 25 06:50:38.259454 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:50:38.259428    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315\": container with ID starting with d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315 not found: ID does not exist" containerID="d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315"
May 25 06:50:38.259562 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:38.259460    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315"} err="failed to get container status \"d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315\": rpc error: code = NotFound desc = could not find container \"d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315\": container with ID starting with d0d8c4a1bee13732e3c9c4de310e0d30cb32236df44e3babd3a5ebfff7c07315 not found: ID does not exist"
May 25 06:50:38.274495 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:38.274464    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-8bf7d7d4-zkxdd"]
May 25 06:50:38.282138 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:38.282115    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-8bf7d7d4-zkxdd"]
May 25 06:50:40.285949 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:50:40.285910    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a5654d3d-84b5-436f-8dbb-05de2dd05478" path="/var/lib/kubelet/pods/a5654d3d-84b5-436f-8dbb-05de2dd05478/volumes"
May 25 06:51:02.210281 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:02.210254    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 06:51:02.212046 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:02.212027    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 06:51:51.939289 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:51.939206    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"]
May 25 06:51:51.939717 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:51.939543    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a5654d3d-84b5-436f-8dbb-05de2dd05478" containerName="console"
May 25 06:51:51.939717 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:51.939557    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a5654d3d-84b5-436f-8dbb-05de2dd05478" containerName="console"
May 25 06:51:51.939717 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:51.939617    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="a5654d3d-84b5-436f-8dbb-05de2dd05478" containerName="console"
May 25 06:51:51.941250 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:51.941233    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"
May 25 06:51:51.945133 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:51.945109    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-d52vl\""
May 25 06:51:51.945807 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:51.945788    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 25 06:51:51.945885 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:51.945825    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 25 06:51:51.962346 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:51.962320    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"]
May 25 06:51:52.071413 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:52.071370    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f8z7j\" (UniqueName: \"kubernetes.io/projected/b4286131-2303-4bed-8c12-7945370edbfa-kube-api-access-f8z7j\") pod \"feast-operator-controller-manager-5bdc49556f-lcbbp\" (UID: \"b4286131-2303-4bed-8c12-7945370edbfa\") " pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"
May 25 06:51:52.172153 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:52.172113    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f8z7j\" (UniqueName: \"kubernetes.io/projected/b4286131-2303-4bed-8c12-7945370edbfa-kube-api-access-f8z7j\") pod \"feast-operator-controller-manager-5bdc49556f-lcbbp\" (UID: \"b4286131-2303-4bed-8c12-7945370edbfa\") " pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"
May 25 06:51:52.183987 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:52.181978    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f8z7j\" (UniqueName: \"kubernetes.io/projected/b4286131-2303-4bed-8c12-7945370edbfa-kube-api-access-f8z7j\") pod \"feast-operator-controller-manager-5bdc49556f-lcbbp\" (UID: \"b4286131-2303-4bed-8c12-7945370edbfa\") " pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"
May 25 06:51:52.251398 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:52.251308    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"
May 25 06:51:52.381648 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:52.381622    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"]
May 25 06:51:52.384224 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:51:52.384190    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podb4286131_2303_4bed_8c12_7945370edbfa.slice/crio-eff5f1ab02126506d3555d65cf140efdf91dffd00f89a6a37fe3751ea50c6bf8 WatchSource:0}: Error finding container eff5f1ab02126506d3555d65cf140efdf91dffd00f89a6a37fe3751ea50c6bf8: Status 404 returned error can't find the container with id eff5f1ab02126506d3555d65cf140efdf91dffd00f89a6a37fe3751ea50c6bf8
May 25 06:51:52.386022 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:52.386005    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 06:51:52.455147 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:52.455107    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp" event={"ID":"b4286131-2303-4bed-8c12-7945370edbfa","Type":"ContainerStarted","Data":"eff5f1ab02126506d3555d65cf140efdf91dffd00f89a6a37fe3751ea50c6bf8"}
May 25 06:51:55.464451 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:55.464412    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp" event={"ID":"b4286131-2303-4bed-8c12-7945370edbfa","Type":"ContainerStarted","Data":"9a3a01db961e50b1b443da2013fd98e6cfa79d77ba35732f0edec14e7a5d07f0"}
May 25 06:51:55.464940 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:55.464545    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"
May 25 06:51:55.482786 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:51:55.482736    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp" podStartSLOduration=1.594449253 podStartE2EDuration="4.48272123s" podCreationTimestamp="2026-05-25 06:51:51 +0000 UTC" firstStartedPulling="2026-05-25 06:51:52.386159297 +0000 UTC m=+350.622345667" lastFinishedPulling="2026-05-25 06:51:55.274431272 +0000 UTC m=+353.510617644" observedRunningTime="2026-05-25 06:51:55.481902957 +0000 UTC m=+353.718089350" watchObservedRunningTime="2026-05-25 06:51:55.48272123 +0000 UTC m=+353.718907623"
May 25 06:52:06.469241 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:06.469209    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"
May 25 06:52:35.766980 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.766948    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"]
May 25 06:52:35.775690 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.775669    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.778181 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.778157    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 25 06:52:35.778894 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.778865    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 25 06:52:35.779023 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.778926    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 25 06:52:35.779023 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.778936    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 25 06:52:35.779023 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.778995    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 25 06:52:35.779151 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.779031    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 25 06:52:35.779383 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.779368    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 25 06:52:35.779492 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.779476    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-bw85k\""
May 25 06:52:35.790111 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.790084    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"]
May 25 06:52:35.816062 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.816031    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-online-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.816239 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.816083    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-ui-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.816239 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.816113    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gz9sg\" (UniqueName: \"kubernetes.io/projected/cfa8be02-ac61-4580-b655-dc0a610e2c4e-kube-api-access-gz9sg\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.816239 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.816138    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-offline-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.816239 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.816155    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa8be02-ac61-4580-b655-dc0a610e2c4e-feast-data\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.816239 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.816180    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-registry-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.916503 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.916473    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-online-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.916699 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.916520    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-ui-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.916699 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:52:35.916606    2572 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 25 06:52:35.916699 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.916657    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gz9sg\" (UniqueName: \"kubernetes.io/projected/cfa8be02-ac61-4580-b655-dc0a610e2c4e-kube-api-access-gz9sg\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.916699 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:52:35.916665    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-ui-tls podName:cfa8be02-ac61-4580-b655-dc0a610e2c4e nodeName:}" failed. No retries permitted until 2026-05-25 06:52:36.41664886 +0000 UTC m=+394.652835230 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-ui-tls") pod "feast-simple-feast-setup-9f6cf4b9b-r67m7" (UID: "cfa8be02-ac61-4580-b655-dc0a610e2c4e") : secret "feast-simple-feast-setup-ui-tls" not found
May 25 06:52:35.916965 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.916727    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-offline-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.916965 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.916766    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa8be02-ac61-4580-b655-dc0a610e2c4e-feast-data\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.916965 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.916797    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-registry-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.916965 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:52:35.916835    2572 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 25 06:52:35.916965 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:52:35.916939    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-offline-tls podName:cfa8be02-ac61-4580-b655-dc0a610e2c4e nodeName:}" failed. No retries permitted until 2026-05-25 06:52:36.416918755 +0000 UTC m=+394.653105140 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-offline-tls") pod "feast-simple-feast-setup-9f6cf4b9b-r67m7" (UID: "cfa8be02-ac61-4580-b655-dc0a610e2c4e") : secret "feast-simple-feast-setup-offline-tls" not found
May 25 06:52:35.917163 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.917103    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa8be02-ac61-4580-b655-dc0a610e2c4e-feast-data\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.919072 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.919052    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-registry-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.919165 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.919122    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-online-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:35.927176 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:35.927144    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gz9sg\" (UniqueName: \"kubernetes.io/projected/cfa8be02-ac61-4580-b655-dc0a610e2c4e-kube-api-access-gz9sg\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:36.421579 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:36.421540    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-ui-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:36.421775 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:36.421604    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-offline-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:36.424160 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:36.424130    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-offline-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:36.424270 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:36.424184    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-ui-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-r67m7\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:36.699867 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:36.699754    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:52:36.836453 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:36.836404    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"]
May 25 06:52:36.841879 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:52:36.841828    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podcfa8be02_ac61_4580_b655_dc0a610e2c4e.slice/crio-bf49a269a792ed928750c53c7d39864227f735e7f4c89d60004bbe6f52580ee7 WatchSource:0}: Error finding container bf49a269a792ed928750c53c7d39864227f735e7f4c89d60004bbe6f52580ee7: Status 404 returned error can't find the container with id bf49a269a792ed928750c53c7d39864227f735e7f4c89d60004bbe6f52580ee7
May 25 06:52:37.582247 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:52:37.582210    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerStarted","Data":"bf49a269a792ed928750c53c7d39864227f735e7f4c89d60004bbe6f52580ee7"}
May 25 06:53:05.668995 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:05.668956    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerStarted","Data":"68c36e7e2807fec2edd89dee35b42751988e788c85a702ac9d462f8cbc9db570"}
May 25 06:53:10.685102 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:10.685066    2572 generic.go:358] "Generic (PLEG): container finished" podID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerID="68c36e7e2807fec2edd89dee35b42751988e788c85a702ac9d462f8cbc9db570" exitCode=0
May 25 06:53:10.685637 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:10.685137    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerDied","Data":"68c36e7e2807fec2edd89dee35b42751988e788c85a702ac9d462f8cbc9db570"}
May 25 06:53:11.689721 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:11.689687    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerStarted","Data":"878e31af1c4b664c8ab180363552a1fe2292c2a994243af7bba7afb26a621a8f"}
May 25 06:53:17.710085 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:17.710048    2572 generic.go:358] "Generic (PLEG): container finished" podID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerID="878e31af1c4b664c8ab180363552a1fe2292c2a994243af7bba7afb26a621a8f" exitCode=0
May 25 06:53:17.710551 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:17.710129    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerDied","Data":"878e31af1c4b664c8ab180363552a1fe2292c2a994243af7bba7afb26a621a8f"}
May 25 06:53:17.719893 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:17.719873    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 25 06:53:18.718407 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:18.717424    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerStarted","Data":"809c4af213f49222bc0cd8976f16f4f142bd34733ba4295491f2d200c569cc22"}
May 25 06:53:18.718407 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:18.717468    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerStarted","Data":"fe0b4ae9162740d8c23558be16d402489790b5df490aaf383e2432ace6dc2d6a"}
May 25 06:53:18.718407 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:18.717483    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerStarted","Data":"d5f7e0489a15b45fcd718b72d08ee2f3a5d30e7627fe996857fb5fae269959e9"}
May 25 06:53:18.718407 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:18.717495    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerStarted","Data":"1e2b30497b3add7f5bc8d67042aa6a382c8290b45ae95cd372a03317eddc8263"}
May 25 06:53:18.741152 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:18.741069    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podStartSLOduration=15.54998978 podStartE2EDuration="43.74104623s" podCreationTimestamp="2026-05-25 06:52:35 +0000 UTC" firstStartedPulling="2026-05-25 06:52:36.843583317 +0000 UTC m=+395.079769687" lastFinishedPulling="2026-05-25 06:53:05.034639752 +0000 UTC m=+423.270826137" observedRunningTime="2026-05-25 06:53:18.738654009 +0000 UTC m=+436.974840401" watchObservedRunningTime="2026-05-25 06:53:18.74104623 +0000 UTC m=+436.977232626"
May 25 06:53:21.700288 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:21.700188    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:21.700288 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:21.700249    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:21.700288 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:21.700265    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:21.700981 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:21.700738    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:21.702278 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:21.702240    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 25 06:53:21.702408 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:21.702278    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 25 06:53:21.702408 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:21.702361    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused"
May 25 06:53:21.702518 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:21.702423    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused"
May 25 06:53:24.701510 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:24.701481    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:24.701914 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:24.701615    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:24.701914 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:24.701890    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:24.702281 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:24.702255    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:24.702400 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:24.702289    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:24.702400 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:24.702301    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:24.702507 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:24.702489    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:24.702626 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:24.702612    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:24.702744 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:24.702729    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:24.705950 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:24.705936    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:24.738097 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:24.738074    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:24.740940 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:24.740916    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:53:43.042924 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.042888    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-dcdb769b6-2sqn8"]
May 25 06:53:43.058177 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.058150    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dcdb769b6-2sqn8"]
May 25 06:53:43.058312 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.058261    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.073302 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.073278    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb56ca3e-99a5-4239-850b-c065b8dc7353-console-oauth-config\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.073438 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.073313    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb56ca3e-99a5-4239-850b-c065b8dc7353-service-ca\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.073438 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.073330    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb56ca3e-99a5-4239-850b-c065b8dc7353-trusted-ca-bundle\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.073438 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.073399    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb56ca3e-99a5-4239-850b-c065b8dc7353-console-config\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.073438 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.073433    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb56ca3e-99a5-4239-850b-c065b8dc7353-console-serving-cert\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.073626 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.073509    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dg7tj\" (UniqueName: \"kubernetes.io/projected/cb56ca3e-99a5-4239-850b-c065b8dc7353-kube-api-access-dg7tj\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.073626 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.073537    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb56ca3e-99a5-4239-850b-c065b8dc7353-oauth-serving-cert\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.173907 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.173870    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb56ca3e-99a5-4239-850b-c065b8dc7353-console-config\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.173907 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.173913    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb56ca3e-99a5-4239-850b-c065b8dc7353-console-serving-cert\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.174143 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.173968    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dg7tj\" (UniqueName: \"kubernetes.io/projected/cb56ca3e-99a5-4239-850b-c065b8dc7353-kube-api-access-dg7tj\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.174143 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.173993    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb56ca3e-99a5-4239-850b-c065b8dc7353-oauth-serving-cert\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.174143 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.174027    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb56ca3e-99a5-4239-850b-c065b8dc7353-console-oauth-config\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.174143 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.174067    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb56ca3e-99a5-4239-850b-c065b8dc7353-service-ca\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.174143 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.174088    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb56ca3e-99a5-4239-850b-c065b8dc7353-trusted-ca-bundle\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.174693 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.174661    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/cb56ca3e-99a5-4239-850b-c065b8dc7353-console-config\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.174812 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.174718    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/cb56ca3e-99a5-4239-850b-c065b8dc7353-oauth-serving-cert\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.174812 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.174776    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/cb56ca3e-99a5-4239-850b-c065b8dc7353-service-ca\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.174999 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.174977    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/cb56ca3e-99a5-4239-850b-c065b8dc7353-trusted-ca-bundle\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.176617 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.176594    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/cb56ca3e-99a5-4239-850b-c065b8dc7353-console-oauth-config\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.176716 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.176603    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/cb56ca3e-99a5-4239-850b-c065b8dc7353-console-serving-cert\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.180827 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.180798    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dg7tj\" (UniqueName: \"kubernetes.io/projected/cb56ca3e-99a5-4239-850b-c065b8dc7353-kube-api-access-dg7tj\") pod \"console-dcdb769b6-2sqn8\" (UID: \"cb56ca3e-99a5-4239-850b-c065b8dc7353\") " pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.367401 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.367363    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:43.493785 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.493745    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-dcdb769b6-2sqn8"]
May 25 06:53:43.496660 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:53:43.496634    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podcb56ca3e_99a5_4239_850b_c065b8dc7353.slice/crio-cc55464b841b3e127d2d99e8611c8beee19f6c02cd78ed3f2bc1ed3f18fff34b WatchSource:0}: Error finding container cc55464b841b3e127d2d99e8611c8beee19f6c02cd78ed3f2bc1ed3f18fff34b: Status 404 returned error can't find the container with id cc55464b841b3e127d2d99e8611c8beee19f6c02cd78ed3f2bc1ed3f18fff34b
May 25 06:53:43.794676 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.794588    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dcdb769b6-2sqn8" event={"ID":"cb56ca3e-99a5-4239-850b-c065b8dc7353","Type":"ContainerStarted","Data":"5003a622d0f0cea2997d3f0fab80b0c4fc57fcbae8ad6749b12b46809f71373d"}
May 25 06:53:43.794676 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.794631    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-dcdb769b6-2sqn8" event={"ID":"cb56ca3e-99a5-4239-850b-c065b8dc7353","Type":"ContainerStarted","Data":"cc55464b841b3e127d2d99e8611c8beee19f6c02cd78ed3f2bc1ed3f18fff34b"}
May 25 06:53:43.814103 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:43.814034    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-dcdb769b6-2sqn8" podStartSLOduration=0.814020951 podStartE2EDuration="814.020951ms" podCreationTimestamp="2026-05-25 06:53:43 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 06:53:43.813973639 +0000 UTC m=+462.050160033" watchObservedRunningTime="2026-05-25 06:53:43.814020951 +0000 UTC m=+462.050207342"
May 25 06:53:53.368148 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:53.368107    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:53.368148 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:53.368158    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:53.372917 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:53.372892    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:53.827647 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:53.827620    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-dcdb769b6-2sqn8"
May 25 06:53:53.873068 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:53:53.873028    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-687cf575c8-ps6n6"]
May 25 06:54:18.897129 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:18.897074    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-687cf575c8-ps6n6" podUID="c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" containerName="console" containerID="cri-o://954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee" gracePeriod=15
May 25 06:54:19.131675 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.131653    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-687cf575c8-ps6n6_c3ba1d94-4e5b-4f2d-bc27-a2a079617a83/console/0.log"
May 25 06:54:19.131883 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.131709    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:54:19.269540 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.269447    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-config\") pod \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") "
May 25 06:54:19.269540 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.269503    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-serving-cert\") pod \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") "
May 25 06:54:19.269540 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.269524    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-oauth-serving-cert\") pod \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") "
May 25 06:54:19.269812 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.269655    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-9xq6t\" (UniqueName: \"kubernetes.io/projected/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-kube-api-access-9xq6t\") pod \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") "
May 25 06:54:19.269812 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.269716    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-oauth-config\") pod \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") "
May 25 06:54:19.269812 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.269757    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-trusted-ca-bundle\") pod \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") "
May 25 06:54:19.269999 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.269812    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-service-ca\") pod \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\" (UID: \"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83\") "
May 25 06:54:19.269999 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.269898    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-config" (OuterVolumeSpecName: "console-config") pod "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" (UID: "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:54:19.269999 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.269906    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" (UID: "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:54:19.270198 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.270179    2572 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-config\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:19.270249 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.270190    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" (UID: "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:54:19.270249 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.270208    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-service-ca" (OuterVolumeSpecName: "service-ca") pod "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" (UID: "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:54:19.270249 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.270200    2572 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-oauth-serving-cert\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:19.271947 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.271928    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-kube-api-access-9xq6t" (OuterVolumeSpecName: "kube-api-access-9xq6t") pod "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" (UID: "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83"). InnerVolumeSpecName "kube-api-access-9xq6t". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 06:54:19.272314 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.272280    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" (UID: "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:54:19.272404 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.272328    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" (UID: "c3ba1d94-4e5b-4f2d-bc27-a2a079617a83"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:54:19.371384 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.371345    2572 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-service-ca\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:19.371384 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.371376    2572 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-serving-cert\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:19.371384 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.371388    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-9xq6t\" (UniqueName: \"kubernetes.io/projected/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-kube-api-access-9xq6t\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:19.371606 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.371399    2572 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-console-oauth-config\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:19.371606 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.371407    2572 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83-trusted-ca-bundle\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:19.898903 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.898879    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-687cf575c8-ps6n6_c3ba1d94-4e5b-4f2d-bc27-a2a079617a83/console/0.log"
May 25 06:54:19.899271 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.898916    2572 generic.go:358] "Generic (PLEG): container finished" podID="c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" containerID="954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee" exitCode=2
May 25 06:54:19.899271 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.898950    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-687cf575c8-ps6n6" event={"ID":"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83","Type":"ContainerDied","Data":"954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee"}
May 25 06:54:19.899271 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.898986    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-687cf575c8-ps6n6" event={"ID":"c3ba1d94-4e5b-4f2d-bc27-a2a079617a83","Type":"ContainerDied","Data":"1a93be3fce2b1d31d438c5abaedb91efb54b3e07f0e371c6320ed3e2a88078df"}
May 25 06:54:19.899271 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.898982    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-687cf575c8-ps6n6"
May 25 06:54:19.899271 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.898997    2572 scope.go:117] "RemoveContainer" containerID="954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee"
May 25 06:54:19.907326 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.907304    2572 scope.go:117] "RemoveContainer" containerID="954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee"
May 25 06:54:19.907623 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:54:19.907602    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee\": container with ID starting with 954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee not found: ID does not exist" containerID="954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee"
May 25 06:54:19.907695 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.907630    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee"} err="failed to get container status \"954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee\": rpc error: code = NotFound desc = could not find container \"954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee\": container with ID starting with 954e76ecc007a48a0bd5db94fe9d98fe89fac12144dcc4bbb7a27af719f824ee not found: ID does not exist"
May 25 06:54:19.918971 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.918943    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-687cf575c8-ps6n6"]
May 25 06:54:19.923782 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:19.923757    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-687cf575c8-ps6n6"]
May 25 06:54:20.287017 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:20.283882    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" path="/var/lib/kubelet/pods/c3ba1d94-4e5b-4f2d-bc27-a2a079617a83/volumes"
May 25 06:54:27.003813 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.003779    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"]
May 25 06:54:27.004471 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.004236    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="registry" containerID="cri-o://1e2b30497b3add7f5bc8d67042aa6a382c8290b45ae95cd372a03317eddc8263" gracePeriod=30
May 25 06:54:27.004555 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.004471    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="ui" containerID="cri-o://809c4af213f49222bc0cd8976f16f4f142bd34733ba4295491f2d200c569cc22" gracePeriod=30
May 25 06:54:27.004611 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.004522    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="online" containerID="cri-o://d5f7e0489a15b45fcd718b72d08ee2f3a5d30e7627fe996857fb5fae269959e9" gracePeriod=30
May 25 06:54:27.004611 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.004595    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="offline" containerID="cri-o://fe0b4ae9162740d8c23558be16d402489790b5df490aaf383e2432ace6dc2d6a" gracePeriod=30
May 25 06:54:27.261113 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.261027    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"]
May 25 06:54:27.261404 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.261386    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" containerName="console"
May 25 06:54:27.261482 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.261410    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" containerName="console"
May 25 06:54:27.261542 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.261502    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="c3ba1d94-4e5b-4f2d-bc27-a2a079617a83" containerName="console"
May 25 06:54:27.264922 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.264899    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.266750 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.266726    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-chq8z\""
May 25 06:54:27.276388 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.276353    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"]
May 25 06:54:27.345689 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.345645    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-offline-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.345689 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.345690    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0782bacf-2540-4526-964e-a71c4d4e8be7-feast-data\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.345976 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.345800    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-online-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.345976 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.345909    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xmb54\" (UniqueName: \"kubernetes.io/projected/0782bacf-2540-4526-964e-a71c4d4e8be7-kube-api-access-xmb54\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.345976 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.345943    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-registry-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.346093 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.346006    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-ui-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.447555 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.447506    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-offline-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.447555 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.447563    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0782bacf-2540-4526-964e-a71c4d4e8be7-feast-data\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.447822 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.447639    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-online-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.447822 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.447699    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xmb54\" (UniqueName: \"kubernetes.io/projected/0782bacf-2540-4526-964e-a71c4d4e8be7-kube-api-access-xmb54\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.447822 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.447729    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-registry-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.447822 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:54:27.447815    2572 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 25 06:54:27.448079 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:54:27.447837    2572 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 25 06:54:27.448079 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:54:27.447896    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-registry-tls podName:0782bacf-2540-4526-964e-a71c4d4e8be7 nodeName:}" failed. No retries permitted until 2026-05-25 06:54:27.947875199 +0000 UTC m=+506.184061583 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-registry-tls") pod "feast-simple-feast-setup-9f6cf4b9b-488hj" (UID: "0782bacf-2540-4526-964e-a71c4d4e8be7") : secret "feast-simple-feast-setup-registry-tls" not found
May 25 06:54:27.448079 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:54:27.447929    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-online-tls podName:0782bacf-2540-4526-964e-a71c4d4e8be7 nodeName:}" failed. No retries permitted until 2026-05-25 06:54:27.947910402 +0000 UTC m=+506.184096777 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-online-tls") pod "feast-simple-feast-setup-9f6cf4b9b-488hj" (UID: "0782bacf-2540-4526-964e-a71c4d4e8be7") : secret "feast-simple-feast-setup-online-tls" not found
May 25 06:54:27.448079 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.447957    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-ui-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.448079 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:54:27.448069    2572 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 25 06:54:27.448594 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.448096    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0782bacf-2540-4526-964e-a71c4d4e8be7-feast-data\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.448594 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:54:27.448113    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-ui-tls podName:0782bacf-2540-4526-964e-a71c4d4e8be7 nodeName:}" failed. No retries permitted until 2026-05-25 06:54:27.948101428 +0000 UTC m=+506.184287798 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-ui-tls") pod "feast-simple-feast-setup-9f6cf4b9b-488hj" (UID: "0782bacf-2540-4526-964e-a71c4d4e8be7") : secret "feast-simple-feast-setup-ui-tls" not found
May 25 06:54:27.450410 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.450381    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-offline-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.455629 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.455602    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xmb54\" (UniqueName: \"kubernetes.io/projected/0782bacf-2540-4526-964e-a71c4d4e8be7-kube-api-access-xmb54\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.925947 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.925916    2572 generic.go:358] "Generic (PLEG): container finished" podID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerID="809c4af213f49222bc0cd8976f16f4f142bd34733ba4295491f2d200c569cc22" exitCode=0
May 25 06:54:27.925947 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.925941    2572 generic.go:358] "Generic (PLEG): container finished" podID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerID="d5f7e0489a15b45fcd718b72d08ee2f3a5d30e7627fe996857fb5fae269959e9" exitCode=0
May 25 06:54:27.926167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.925990    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerDied","Data":"809c4af213f49222bc0cd8976f16f4f142bd34733ba4295491f2d200c569cc22"}
May 25 06:54:27.926167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.926034    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerDied","Data":"d5f7e0489a15b45fcd718b72d08ee2f3a5d30e7627fe996857fb5fae269959e9"}
May 25 06:54:27.953407 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.953365    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-registry-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.953407 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.953410    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-ui-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.953674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.953481    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-online-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.955878 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.955833    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-online-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.956046 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.956026    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-registry-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:27.956107 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:27.956028    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-ui-tls\") pod \"feast-simple-feast-setup-9f6cf4b9b-488hj\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") " pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:28.174903 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:28.174837    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:28.313903 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:28.313865    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"]
May 25 06:54:28.316983 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:54:28.316957    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod0782bacf_2540_4526_964e_a71c4d4e8be7.slice/crio-aa99a5cb508e7aa183d66cdd247502daf83a3d6b4dbdcfa0750cab92a38a7f2d WatchSource:0}: Error finding container aa99a5cb508e7aa183d66cdd247502daf83a3d6b4dbdcfa0750cab92a38a7f2d: Status 404 returned error can't find the container with id aa99a5cb508e7aa183d66cdd247502daf83a3d6b4dbdcfa0750cab92a38a7f2d
May 25 06:54:28.930880 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:28.930823    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerStarted","Data":"8f39f88bc9ca202b978d449be686eaeece48220a2570579560354c62e0b49984"}
May 25 06:54:28.930880 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:28.930882    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerStarted","Data":"aa99a5cb508e7aa183d66cdd247502daf83a3d6b4dbdcfa0750cab92a38a7f2d"}
May 25 06:54:33.948260 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:33.948224    2572 generic.go:358] "Generic (PLEG): container finished" podID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerID="8f39f88bc9ca202b978d449be686eaeece48220a2570579560354c62e0b49984" exitCode=0
May 25 06:54:33.948628 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:33.948298    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerDied","Data":"8f39f88bc9ca202b978d449be686eaeece48220a2570579560354c62e0b49984"}
May 25 06:54:34.703138 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:34.703091    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 25 06:54:34.739154 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:34.739115    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 25 06:54:34.953989 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:34.953909    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerStarted","Data":"182b39cc36ab1fdf0da43e15f6ca345a16f9a529812597e2e05315f5d76e6b3b"}
May 25 06:54:39.973352 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:39.973314    2572 generic.go:358] "Generic (PLEG): container finished" podID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerID="182b39cc36ab1fdf0da43e15f6ca345a16f9a529812597e2e05315f5d76e6b3b" exitCode=0
May 25 06:54:39.973757 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:39.973380    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerDied","Data":"182b39cc36ab1fdf0da43e15f6ca345a16f9a529812597e2e05315f5d76e6b3b"}
May 25 06:54:40.981791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:40.981751    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerStarted","Data":"e9a1e11a9780e295c21eed02b83c90f6a7d424cab9f3909c18f603472ce2f9ec"}
May 25 06:54:40.982299 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:40.981803    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerStarted","Data":"0009f5aac67dfa956f0fa111eba0e1ada788d9df397cc7b964bc24a4202326cd"}
May 25 06:54:40.982299 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:40.981822    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerStarted","Data":"5ba8908377b078897c6a6d78355f0b96287ccc8bf5167b773a9c8c8c6b1fdb08"}
May 25 06:54:40.982299 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:40.981836    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerStarted","Data":"37896051f25da06967ccfc9003245d45c83feb52e068c5c187f0f249768feb55"}
May 25 06:54:41.004930 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:41.004836    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podStartSLOduration=14.004816155 podStartE2EDuration="14.004816155s" podCreationTimestamp="2026-05-25 06:54:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 06:54:41.004014204 +0000 UTC m=+519.240200597" watchObservedRunningTime="2026-05-25 06:54:41.004816155 +0000 UTC m=+519.241002547"
May 25 06:54:43.175526 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:43.175478    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:43.175526 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:43.175537    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:43.176120 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:43.175551    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:43.176120 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:43.175565    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:43.178023 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:43.177894    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 25 06:54:43.178023 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:43.177917    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 25 06:54:43.178023 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:43.177939    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused"
May 25 06:54:43.178023 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:43.177896    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.26:6571: connect: connection refused"
May 25 06:54:44.702865 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:44.702518    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 25 06:54:44.738377 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:44.738330    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 25 06:54:46.176209 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:46.176152    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:46.176676 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:46.176315    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:46.176676 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:46.176629    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:46.176676 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:46.176661    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:46.176676 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:46.176673    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:46.176972 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:46.176684    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:46.177032 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:46.176987    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:46.177125 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:46.177106    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:46.177174 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:46.177149    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:46.181144 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:46.181121    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:47.010913 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:47.010882    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:47.014060 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:47.014038    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:54:54.702592 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:54.702482    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 25 06:54:54.703059 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:54.702631    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:54:54.738777 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:54.738735    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 25 06:54:54.738973 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:54.738878    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:54:57.041785 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.041756    2572 generic.go:358] "Generic (PLEG): container finished" podID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerID="fe0b4ae9162740d8c23558be16d402489790b5df490aaf383e2432ace6dc2d6a" exitCode=137
May 25 06:54:57.041785 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.041780    2572 generic.go:358] "Generic (PLEG): container finished" podID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerID="1e2b30497b3add7f5bc8d67042aa6a382c8290b45ae95cd372a03317eddc8263" exitCode=137
May 25 06:54:57.042178 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.041828    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerDied","Data":"fe0b4ae9162740d8c23558be16d402489790b5df490aaf383e2432ace6dc2d6a"}
May 25 06:54:57.042178 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.041893    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerDied","Data":"1e2b30497b3add7f5bc8d67042aa6a382c8290b45ae95cd372a03317eddc8263"}
May 25 06:54:57.656471 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.656442    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:54:57.701078 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.701049    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-online-tls\") pod \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") "
May 25 06:54:57.701078 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.701090    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-ui-tls\") pod \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") "
May 25 06:54:57.701334 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.701119    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-offline-tls\") pod \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") "
May 25 06:54:57.701334 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.701151    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa8be02-ac61-4580-b655-dc0a610e2c4e-feast-data\") pod \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") "
May 25 06:54:57.701334 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.701169    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gz9sg\" (UniqueName: \"kubernetes.io/projected/cfa8be02-ac61-4580-b655-dc0a610e2c4e-kube-api-access-gz9sg\") pod \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") "
May 25 06:54:57.701334 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.701262    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-registry-tls\") pod \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\" (UID: \"cfa8be02-ac61-4580-b655-dc0a610e2c4e\") "
May 25 06:54:57.701773 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.701731    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/cfa8be02-ac61-4580-b655-dc0a610e2c4e-feast-data" (OuterVolumeSpecName: "feast-data") pod "cfa8be02-ac61-4580-b655-dc0a610e2c4e" (UID: "cfa8be02-ac61-4580-b655-dc0a610e2c4e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 06:54:57.703431 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.703409    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-online-tls" (OuterVolumeSpecName: "online-tls") pod "cfa8be02-ac61-4580-b655-dc0a610e2c4e" (UID: "cfa8be02-ac61-4580-b655-dc0a610e2c4e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:54:57.703539 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.703432    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "cfa8be02-ac61-4580-b655-dc0a610e2c4e" (UID: "cfa8be02-ac61-4580-b655-dc0a610e2c4e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:54:57.703539 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.703443    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "cfa8be02-ac61-4580-b655-dc0a610e2c4e" (UID: "cfa8be02-ac61-4580-b655-dc0a610e2c4e"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:54:57.703714 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.703697    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "cfa8be02-ac61-4580-b655-dc0a610e2c4e" (UID: "cfa8be02-ac61-4580-b655-dc0a610e2c4e"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:54:57.703815 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.703797    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/cfa8be02-ac61-4580-b655-dc0a610e2c4e-kube-api-access-gz9sg" (OuterVolumeSpecName: "kube-api-access-gz9sg") pod "cfa8be02-ac61-4580-b655-dc0a610e2c4e" (UID: "cfa8be02-ac61-4580-b655-dc0a610e2c4e"). InnerVolumeSpecName "kube-api-access-gz9sg". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 06:54:57.802200 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.802097    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:57.802200 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.802143    2572 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-online-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:57.802200 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.802152    2572 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-ui-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:57.802200 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.802160    2572 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/cfa8be02-ac61-4580-b655-dc0a610e2c4e-offline-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:57.802200 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.802168    2572 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/cfa8be02-ac61-4580-b655-dc0a610e2c4e-feast-data\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:57.802200 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:57.802177    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gz9sg\" (UniqueName: \"kubernetes.io/projected/cfa8be02-ac61-4580-b655-dc0a610e2c4e-kube-api-access-gz9sg\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:54:58.046904 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:58.046876    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"
May 25 06:54:58.046904 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:58.046876    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7" event={"ID":"cfa8be02-ac61-4580-b655-dc0a610e2c4e","Type":"ContainerDied","Data":"bf49a269a792ed928750c53c7d39864227f735e7f4c89d60004bbe6f52580ee7"}
May 25 06:54:58.047389 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:58.046928    2572 scope.go:117] "RemoveContainer" containerID="809c4af213f49222bc0cd8976f16f4f142bd34733ba4295491f2d200c569cc22"
May 25 06:54:58.055485 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:58.055453    2572 scope.go:117] "RemoveContainer" containerID="fe0b4ae9162740d8c23558be16d402489790b5df490aaf383e2432ace6dc2d6a"
May 25 06:54:58.066459 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:58.066438    2572 scope.go:117] "RemoveContainer" containerID="d5f7e0489a15b45fcd718b72d08ee2f3a5d30e7627fe996857fb5fae269959e9"
May 25 06:54:58.069572 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:58.069550    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"]
May 25 06:54:58.073771 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:58.073746    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-r67m7"]
May 25 06:54:58.075428 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:58.075411    2572 scope.go:117] "RemoveContainer" containerID="1e2b30497b3add7f5bc8d67042aa6a382c8290b45ae95cd372a03317eddc8263"
May 25 06:54:58.082651 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:58.082629    2572 scope.go:117] "RemoveContainer" containerID="878e31af1c4b664c8ab180363552a1fe2292c2a994243af7bba7afb26a621a8f"
May 25 06:54:58.089764 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:58.089747    2572 scope.go:117] "RemoveContainer" containerID="68c36e7e2807fec2edd89dee35b42751988e788c85a702ac9d462f8cbc9db570"
May 25 06:54:58.282498 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:54:58.282462    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" path="/var/lib/kubelet/pods/cfa8be02-ac61-4580-b655-dc0a610e2c4e/volumes"
May 25 06:55:48.992385 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992352    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"]
May 25 06:55:48.992958 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992772    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="offline"
May 25 06:55:48.992958 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992791    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="offline"
May 25 06:55:48.992958 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992803    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="ui"
May 25 06:55:48.992958 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992811    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="ui"
May 25 06:55:48.992958 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992868    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="online"
May 25 06:55:48.992958 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992878    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="online"
May 25 06:55:48.992958 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992888    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="feast-apply"
May 25 06:55:48.992958 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992897    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="feast-apply"
May 25 06:55:48.992958 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992909    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="registry"
May 25 06:55:48.992958 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992917    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="registry"
May 25 06:55:48.992958 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992931    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="feast-init"
May 25 06:55:48.992958 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.992939    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="feast-init"
May 25 06:55:48.993556 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.993038    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="offline"
May 25 06:55:48.993556 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.993053    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="ui"
May 25 06:55:48.993556 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.993064    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="online"
May 25 06:55:48.993556 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.993074    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="cfa8be02-ac61-4580-b655-dc0a610e2c4e" containerName="registry"
May 25 06:55:48.996643 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.996623    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:48.998516 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.998490    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
May 25 06:55:48.998516 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.998512    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 25 06:55:48.998711 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.998556    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-xql8x\""
May 25 06:55:48.998768 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.998716    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 25 06:55:48.998768 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.998766    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 25 06:55:48.998884 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.998799    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 25 06:55:48.998884 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.998799    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 25 06:55:48.999016 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:48.999001    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 25 06:55:49.006304 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.006283    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"]
May 25 06:55:49.027165 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.027127    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-r5gs6\" (UniqueName: \"kubernetes.io/projected/a25d3e1d-f8a6-4a20-970e-05c6855a292e-kube-api-access-r5gs6\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.027337 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.027174    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a25d3e1d-f8a6-4a20-970e-05c6855a292e-feast-data\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.027337 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.027239    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-offline-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.027466 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.027348    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-ui-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.027466 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.027381    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-online-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.027466 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.027429    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a25d3e1d-f8a6-4a20-970e-05c6855a292e-registry-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.128338 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.128295    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-ui-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.128338 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.128339    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-online-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.128594 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.128377    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a25d3e1d-f8a6-4a20-970e-05c6855a292e-registry-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.128594 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.128403    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-r5gs6\" (UniqueName: \"kubernetes.io/projected/a25d3e1d-f8a6-4a20-970e-05c6855a292e-kube-api-access-r5gs6\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.128594 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.128434    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a25d3e1d-f8a6-4a20-970e-05c6855a292e-feast-data\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.128594 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:55:49.128439    2572 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found
May 25 06:55:49.128594 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.128463    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-offline-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.128594 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:55:49.128517    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-ui-tls podName:a25d3e1d-f8a6-4a20-970e-05c6855a292e nodeName:}" failed. No retries permitted until 2026-05-25 06:55:49.628495044 +0000 UTC m=+587.864681423 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-ui-tls") pod "feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" (UID: "a25d3e1d-f8a6-4a20-970e-05c6855a292e") : secret "feast-simple-feast-remote-setup-ui-tls" not found
May 25 06:55:49.128594 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:55:49.128571    2572 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 25 06:55:49.129010 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:55:49.128629    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-offline-tls podName:a25d3e1d-f8a6-4a20-970e-05c6855a292e nodeName:}" failed. No retries permitted until 2026-05-25 06:55:49.628611854 +0000 UTC m=+587.864798223 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-offline-tls") pod "feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" (UID: "a25d3e1d-f8a6-4a20-970e-05c6855a292e") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 25 06:55:49.129010 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.128961    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a25d3e1d-f8a6-4a20-970e-05c6855a292e-feast-data\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.129202 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.129182    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a25d3e1d-f8a6-4a20-970e-05c6855a292e-registry-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.130892 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.130875    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-online-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.136122 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.136097    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-r5gs6\" (UniqueName: \"kubernetes.io/projected/a25d3e1d-f8a6-4a20-970e-05c6855a292e-kube-api-access-r5gs6\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.633065 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.633019    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-ui-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.633288 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.633106    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-offline-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.635592 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.635564    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-ui-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.635720 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.635608    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-offline-tls\") pod \"feast-simple-feast-remote-setup-7d978cd9b6-z8b8g\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:49.906636 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:49.906551    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:55:50.040394 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:50.040373    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"]
May 25 06:55:50.042544 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:55:50.042509    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda25d3e1d_f8a6_4a20_970e_05c6855a292e.slice/crio-b583131f3740c3e0f245c2c6195f7ecad349b34f7251d4f8d5726fe466d3811d WatchSource:0}: Error finding container b583131f3740c3e0f245c2c6195f7ecad349b34f7251d4f8d5726fe466d3811d: Status 404 returned error can't find the container with id b583131f3740c3e0f245c2c6195f7ecad349b34f7251d4f8d5726fe466d3811d
May 25 06:55:50.200342 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:50.200260    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" event={"ID":"a25d3e1d-f8a6-4a20-970e-05c6855a292e","Type":"ContainerStarted","Data":"8c19e53e3afbd77e26b6457baf576666e8cfc70bf948ff3e57096d796f3f5b33"}
May 25 06:55:50.200342 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:50.200299    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" event={"ID":"a25d3e1d-f8a6-4a20-970e-05c6855a292e","Type":"ContainerStarted","Data":"b583131f3740c3e0f245c2c6195f7ecad349b34f7251d4f8d5726fe466d3811d"}
May 25 06:55:55.216972 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:55.216937    2572 generic.go:358] "Generic (PLEG): container finished" podID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerID="8c19e53e3afbd77e26b6457baf576666e8cfc70bf948ff3e57096d796f3f5b33" exitCode=0
May 25 06:55:55.217332 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:55.217017    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" event={"ID":"a25d3e1d-f8a6-4a20-970e-05c6855a292e","Type":"ContainerDied","Data":"8c19e53e3afbd77e26b6457baf576666e8cfc70bf948ff3e57096d796f3f5b33"}
May 25 06:55:56.221866 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:55:56.221812    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" event={"ID":"a25d3e1d-f8a6-4a20-970e-05c6855a292e","Type":"ContainerStarted","Data":"3de0eaafff149d56365e265dde114f27e2045a98a678e6aa266ca110d240f0ed"}
May 25 06:56:01.237126 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:01.237043    2572 generic.go:358] "Generic (PLEG): container finished" podID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerID="3de0eaafff149d56365e265dde114f27e2045a98a678e6aa266ca110d240f0ed" exitCode=0
May 25 06:56:01.237126 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:01.237090    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" event={"ID":"a25d3e1d-f8a6-4a20-970e-05c6855a292e","Type":"ContainerDied","Data":"3de0eaafff149d56365e265dde114f27e2045a98a678e6aa266ca110d240f0ed"}
May 25 06:56:02.239032 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:02.239000    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 06:56:02.241664 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:02.241637    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 06:56:02.242814 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:02.242787    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" event={"ID":"a25d3e1d-f8a6-4a20-970e-05c6855a292e","Type":"ContainerStarted","Data":"dc51dffad760c41bf02bad87054ee66c6ead3ebe3be40f84f3694bc65f244b09"}
May 25 06:56:02.242948 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:02.242833    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" event={"ID":"a25d3e1d-f8a6-4a20-970e-05c6855a292e","Type":"ContainerStarted","Data":"c0da9360c995f95f8bb83be1b3087dbaf8c17ee93456c6897a087e261eeec407"}
May 25 06:56:02.242948 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:02.242867    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" event={"ID":"a25d3e1d-f8a6-4a20-970e-05c6855a292e","Type":"ContainerStarted","Data":"424553eabc7c966de9dbb52cf3484ca99155a2c8d6adc1fb69c9c9ea284dbfdc"}
May 25 06:56:02.263253 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:02.263189    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podStartSLOduration=14.263174631 podStartE2EDuration="14.263174631s" podCreationTimestamp="2026-05-25 06:55:48 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 06:56:02.262868185 +0000 UTC m=+600.499054580" watchObservedRunningTime="2026-05-25 06:56:02.263174631 +0000 UTC m=+600.499361022"
May 25 06:56:04.907079 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:04.907042    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:56:04.907565 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:04.907120    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:56:04.907565 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:04.907148    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:56:04.908965 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:04.908812    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.27:8816: connect: connection refused"
May 25 06:56:04.908965 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:04.908863    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 25 06:56:04.908965 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:04.908882    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 25 06:56:07.908678 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:07.908645    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:56:07.909111 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:07.908775    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:56:07.909169 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:07.909109    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:56:07.909169 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:07.909143    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:56:07.909468 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:07.909447    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:56:07.909681 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:07.909668    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:56:07.912699 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:07.912684    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:56:08.262972 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:08.262885    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:56:08.265821 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:56:08.265796    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:57:10.291857 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:10.291804    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"]
May 25 06:57:10.292418 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:10.292235    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="online" containerID="cri-o://424553eabc7c966de9dbb52cf3484ca99155a2c8d6adc1fb69c9c9ea284dbfdc" gracePeriod=30
May 25 06:57:10.292418 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:10.292310    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="offline" containerID="cri-o://c0da9360c995f95f8bb83be1b3087dbaf8c17ee93456c6897a087e261eeec407" gracePeriod=30
May 25 06:57:10.292418 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:10.292310    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="ui" containerID="cri-o://dc51dffad760c41bf02bad87054ee66c6ead3ebe3be40f84f3694bc65f244b09" gracePeriod=30
May 25 06:57:10.407981 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:10.407948    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"]
May 25 06:57:10.408465 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:10.408316    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="registry" containerID="cri-o://37896051f25da06967ccfc9003245d45c83feb52e068c5c187f0f249768feb55" gracePeriod=30
May 25 06:57:10.408465 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:10.408353    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="offline" containerID="cri-o://0009f5aac67dfa956f0fa111eba0e1ada788d9df397cc7b964bc24a4202326cd" gracePeriod=30
May 25 06:57:10.408465 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:10.408371    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="ui" containerID="cri-o://e9a1e11a9780e295c21eed02b83c90f6a7d424cab9f3909c18f603472ce2f9ec" gracePeriod=30
May 25 06:57:10.408465 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:10.408358    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="online" containerID="cri-o://5ba8908377b078897c6a6d78355f0b96287ccc8bf5167b773a9c8c8c6b1fdb08" gracePeriod=30
May 25 06:57:11.443380 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:11.443298    2572 generic.go:358] "Generic (PLEG): container finished" podID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerID="e9a1e11a9780e295c21eed02b83c90f6a7d424cab9f3909c18f603472ce2f9ec" exitCode=0
May 25 06:57:11.443380 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:11.443324    2572 generic.go:358] "Generic (PLEG): container finished" podID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerID="5ba8908377b078897c6a6d78355f0b96287ccc8bf5167b773a9c8c8c6b1fdb08" exitCode=0
May 25 06:57:11.443894 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:11.443371    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerDied","Data":"e9a1e11a9780e295c21eed02b83c90f6a7d424cab9f3909c18f603472ce2f9ec"}
May 25 06:57:11.443894 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:11.443407    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerDied","Data":"5ba8908377b078897c6a6d78355f0b96287ccc8bf5167b773a9c8c8c6b1fdb08"}
May 25 06:57:11.445450 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:11.445428    2572 generic.go:358] "Generic (PLEG): container finished" podID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerID="dc51dffad760c41bf02bad87054ee66c6ead3ebe3be40f84f3694bc65f244b09" exitCode=0
May 25 06:57:11.445450 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:11.445447    2572 generic.go:358] "Generic (PLEG): container finished" podID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerID="424553eabc7c966de9dbb52cf3484ca99155a2c8d6adc1fb69c9c9ea284dbfdc" exitCode=0
May 25 06:57:11.445596 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:11.445481    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" event={"ID":"a25d3e1d-f8a6-4a20-970e-05c6855a292e","Type":"ContainerDied","Data":"dc51dffad760c41bf02bad87054ee66c6ead3ebe3be40f84f3694bc65f244b09"}
May 25 06:57:11.445596 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:11.445512    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" event={"ID":"a25d3e1d-f8a6-4a20-970e-05c6855a292e","Type":"ContainerDied","Data":"424553eabc7c966de9dbb52cf3484ca99155a2c8d6adc1fb69c9c9ea284dbfdc"}
May 25 06:57:16.177141 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:16.177089    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 25 06:57:17.012040 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:17.011996    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 25 06:57:17.909799 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:17.909755    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 25 06:57:18.263978 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:18.263886    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 25 06:57:26.177830 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:26.177779    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 25 06:57:27.011929 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:27.011887    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 25 06:57:27.909834 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:27.909792    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 25 06:57:28.263756 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:28.263665    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 25 06:57:36.177096 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:36.177048    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 25 06:57:36.177539 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:36.177184    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:57:37.011417 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:37.011376    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 25 06:57:37.011577 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:37.011505    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:57:37.909460 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:37.909418    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.27:8443: connect: connection refused"
May 25 06:57:37.909932 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:37.909544    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:57:38.263300 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:38.263206    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="online" probeResult="failure" output="Get \"https://10.134.0.27:6567/health\": dial tcp 10.134.0.27:6567: connect: connection refused"
May 25 06:57:38.263453 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:38.263319    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:57:40.531702 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.531669    2572 generic.go:358] "Generic (PLEG): container finished" podID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerID="c0da9360c995f95f8bb83be1b3087dbaf8c17ee93456c6897a087e261eeec407" exitCode=137
May 25 06:57:40.532118 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.531734    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" event={"ID":"a25d3e1d-f8a6-4a20-970e-05c6855a292e","Type":"ContainerDied","Data":"c0da9360c995f95f8bb83be1b3087dbaf8c17ee93456c6897a087e261eeec407"}
May 25 06:57:40.534378 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.534355    2572 generic.go:358] "Generic (PLEG): container finished" podID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerID="0009f5aac67dfa956f0fa111eba0e1ada788d9df397cc7b964bc24a4202326cd" exitCode=137
May 25 06:57:40.534378 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.534374    2572 generic.go:358] "Generic (PLEG): container finished" podID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerID="37896051f25da06967ccfc9003245d45c83feb52e068c5c187f0f249768feb55" exitCode=137
May 25 06:57:40.534553 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.534399    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerDied","Data":"0009f5aac67dfa956f0fa111eba0e1ada788d9df397cc7b964bc24a4202326cd"}
May 25 06:57:40.534553 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.534419    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerDied","Data":"37896051f25da06967ccfc9003245d45c83feb52e068c5c187f0f249768feb55"}
May 25 06:57:40.942674 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.942650    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:57:40.983219 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.983159    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-r5gs6\" (UniqueName: \"kubernetes.io/projected/a25d3e1d-f8a6-4a20-970e-05c6855a292e-kube-api-access-r5gs6\") pod \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") "
May 25 06:57:40.983219 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.983207    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-offline-tls\") pod \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") "
May 25 06:57:40.983581 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.983255    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a25d3e1d-f8a6-4a20-970e-05c6855a292e-registry-tls\") pod \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") "
May 25 06:57:40.983581 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.983295    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a25d3e1d-f8a6-4a20-970e-05c6855a292e-feast-data\") pod \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") "
May 25 06:57:40.983581 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.983528    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-online-tls\") pod \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") "
May 25 06:57:40.983581 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.983560    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-ui-tls\") pod \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\" (UID: \"a25d3e1d-f8a6-4a20-970e-05c6855a292e\") "
May 25 06:57:40.983824 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.983799    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a25d3e1d-f8a6-4a20-970e-05c6855a292e-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a25d3e1d-f8a6-4a20-970e-05c6855a292e" (UID: "a25d3e1d-f8a6-4a20-970e-05c6855a292e"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 06:57:40.983922 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.983887    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a25d3e1d-f8a6-4a20-970e-05c6855a292e-feast-data" (OuterVolumeSpecName: "feast-data") pod "a25d3e1d-f8a6-4a20-970e-05c6855a292e" (UID: "a25d3e1d-f8a6-4a20-970e-05c6855a292e"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 06:57:40.985752 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.985720    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-online-tls" (OuterVolumeSpecName: "online-tls") pod "a25d3e1d-f8a6-4a20-970e-05c6855a292e" (UID: "a25d3e1d-f8a6-4a20-970e-05c6855a292e"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:57:40.985752 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.985738    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "a25d3e1d-f8a6-4a20-970e-05c6855a292e" (UID: "a25d3e1d-f8a6-4a20-970e-05c6855a292e"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:57:40.985933 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.985785    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a25d3e1d-f8a6-4a20-970e-05c6855a292e-kube-api-access-r5gs6" (OuterVolumeSpecName: "kube-api-access-r5gs6") pod "a25d3e1d-f8a6-4a20-970e-05c6855a292e" (UID: "a25d3e1d-f8a6-4a20-970e-05c6855a292e"). InnerVolumeSpecName "kube-api-access-r5gs6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 06:57:40.986137 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:40.986116    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a25d3e1d-f8a6-4a20-970e-05c6855a292e" (UID: "a25d3e1d-f8a6-4a20-970e-05c6855a292e"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:57:41.059057 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.058998    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:57:41.084262 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084238    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-online-tls\") pod \"0782bacf-2540-4526-964e-a71c4d4e8be7\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") "
May 25 06:57:41.084434 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084305    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-registry-tls\") pod \"0782bacf-2540-4526-964e-a71c4d4e8be7\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") "
May 25 06:57:41.084434 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084332    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0782bacf-2540-4526-964e-a71c4d4e8be7-feast-data\") pod \"0782bacf-2540-4526-964e-a71c4d4e8be7\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") "
May 25 06:57:41.084434 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084377    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xmb54\" (UniqueName: \"kubernetes.io/projected/0782bacf-2540-4526-964e-a71c4d4e8be7-kube-api-access-xmb54\") pod \"0782bacf-2540-4526-964e-a71c4d4e8be7\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") "
May 25 06:57:41.084434 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084410    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-ui-tls\") pod \"0782bacf-2540-4526-964e-a71c4d4e8be7\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") "
May 25 06:57:41.084646 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084453    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-offline-tls\") pod \"0782bacf-2540-4526-964e-a71c4d4e8be7\" (UID: \"0782bacf-2540-4526-964e-a71c4d4e8be7\") "
May 25 06:57:41.084705 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084662    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-r5gs6\" (UniqueName: \"kubernetes.io/projected/a25d3e1d-f8a6-4a20-970e-05c6855a292e-kube-api-access-r5gs6\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:57:41.084705 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084682    2572 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-offline-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:57:41.084705 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084701    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a25d3e1d-f8a6-4a20-970e-05c6855a292e-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:57:41.084924 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084717    2572 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a25d3e1d-f8a6-4a20-970e-05c6855a292e-feast-data\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:57:41.084924 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084729    2572 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-online-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:57:41.084924 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084740    2572 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a25d3e1d-f8a6-4a20-970e-05c6855a292e-ui-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:57:41.085120 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.084912    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/0782bacf-2540-4526-964e-a71c4d4e8be7-feast-data" (OuterVolumeSpecName: "feast-data") pod "0782bacf-2540-4526-964e-a71c4d4e8be7" (UID: "0782bacf-2540-4526-964e-a71c4d4e8be7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 06:57:41.086638 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.086601    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "0782bacf-2540-4526-964e-a71c4d4e8be7" (UID: "0782bacf-2540-4526-964e-a71c4d4e8be7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:57:41.086638 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.086619    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/0782bacf-2540-4526-964e-a71c4d4e8be7-kube-api-access-xmb54" (OuterVolumeSpecName: "kube-api-access-xmb54") pod "0782bacf-2540-4526-964e-a71c4d4e8be7" (UID: "0782bacf-2540-4526-964e-a71c4d4e8be7"). InnerVolumeSpecName "kube-api-access-xmb54". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 06:57:41.086872 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.086822    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-online-tls" (OuterVolumeSpecName: "online-tls") pod "0782bacf-2540-4526-964e-a71c4d4e8be7" (UID: "0782bacf-2540-4526-964e-a71c4d4e8be7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:57:41.086872 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.086862    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "0782bacf-2540-4526-964e-a71c4d4e8be7" (UID: "0782bacf-2540-4526-964e-a71c4d4e8be7"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:57:41.087305 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.087281    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "0782bacf-2540-4526-964e-a71c4d4e8be7" (UID: "0782bacf-2540-4526-964e-a71c4d4e8be7"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:57:41.185302 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.185268    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:57:41.185302 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.185297    2572 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/0782bacf-2540-4526-964e-a71c4d4e8be7-feast-data\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:57:41.185302 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.185307    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xmb54\" (UniqueName: \"kubernetes.io/projected/0782bacf-2540-4526-964e-a71c4d4e8be7-kube-api-access-xmb54\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:57:41.185535 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.185319    2572 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-ui-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:57:41.185535 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.185328    2572 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-offline-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:57:41.185535 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.185337    2572 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/0782bacf-2540-4526-964e-a71c4d4e8be7-online-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:57:41.540344 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.540310    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj" event={"ID":"0782bacf-2540-4526-964e-a71c4d4e8be7","Type":"ContainerDied","Data":"aa99a5cb508e7aa183d66cdd247502daf83a3d6b4dbdcfa0750cab92a38a7f2d"}
May 25 06:57:41.540344 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.540332    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"
May 25 06:57:41.540871 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.540359    2572 scope.go:117] "RemoveContainer" containerID="e9a1e11a9780e295c21eed02b83c90f6a7d424cab9f3909c18f603472ce2f9ec"
May 25 06:57:41.542827 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.542802    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g" event={"ID":"a25d3e1d-f8a6-4a20-970e-05c6855a292e","Type":"ContainerDied","Data":"b583131f3740c3e0f245c2c6195f7ecad349b34f7251d4f8d5726fe466d3811d"}
May 25 06:57:41.542827 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.542828    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"
May 25 06:57:41.550832 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.550816    2572 scope.go:117] "RemoveContainer" containerID="0009f5aac67dfa956f0fa111eba0e1ada788d9df397cc7b964bc24a4202326cd"
May 25 06:57:41.558003 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.557984    2572 scope.go:117] "RemoveContainer" containerID="5ba8908377b078897c6a6d78355f0b96287ccc8bf5167b773a9c8c8c6b1fdb08"
May 25 06:57:41.561899 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.561811    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"]
May 25 06:57:41.564121 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.564098    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-9f6cf4b9b-488hj"]
May 25 06:57:41.567448 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.567433    2572 scope.go:117] "RemoveContainer" containerID="37896051f25da06967ccfc9003245d45c83feb52e068c5c187f0f249768feb55"
May 25 06:57:41.573766 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.573743    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"]
May 25 06:57:41.574680 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.574665    2572 scope.go:117] "RemoveContainer" containerID="182b39cc36ab1fdf0da43e15f6ca345a16f9a529812597e2e05315f5d76e6b3b"
May 25 06:57:41.576538 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.576520    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7d978cd9b6-z8b8g"]
May 25 06:57:41.581811 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.581794    2572 scope.go:117] "RemoveContainer" containerID="8f39f88bc9ca202b978d449be686eaeece48220a2570579560354c62e0b49984"
May 25 06:57:41.588584 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.588568    2572 scope.go:117] "RemoveContainer" containerID="dc51dffad760c41bf02bad87054ee66c6ead3ebe3be40f84f3694bc65f244b09"
May 25 06:57:41.595832 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.595816    2572 scope.go:117] "RemoveContainer" containerID="c0da9360c995f95f8bb83be1b3087dbaf8c17ee93456c6897a087e261eeec407"
May 25 06:57:41.602518 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.602502    2572 scope.go:117] "RemoveContainer" containerID="424553eabc7c966de9dbb52cf3484ca99155a2c8d6adc1fb69c9c9ea284dbfdc"
May 25 06:57:41.609259 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.609242    2572 scope.go:117] "RemoveContainer" containerID="3de0eaafff149d56365e265dde114f27e2045a98a678e6aa266ca110d240f0ed"
May 25 06:57:41.616332 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:41.616316    2572 scope.go:117] "RemoveContainer" containerID="8c19e53e3afbd77e26b6457baf576666e8cfc70bf948ff3e57096d796f3f5b33"
May 25 06:57:42.283008 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:42.282974    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" path="/var/lib/kubelet/pods/0782bacf-2540-4526-964e-a71c4d4e8be7/volumes"
May 25 06:57:42.283636 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:42.283622    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" path="/var/lib/kubelet/pods/a25d3e1d-f8a6-4a20-970e-05c6855a292e/volumes"
May 25 06:57:53.300811 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.300736    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-fqcnr"]
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301049    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="feast-init"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301061    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="feast-init"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301076    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="feast-apply"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301081    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="feast-apply"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301088    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="feast-apply"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301093    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="feast-apply"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301101    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="ui"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301106    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="ui"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301113    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="ui"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301119    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="ui"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301126    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="offline"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301130    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="offline"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301136    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="feast-init"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301141    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="feast-init"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301147    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="offline"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301152    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="offline"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301158    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="registry"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301162    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="registry"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301167    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="online"
May 25 06:57:53.301167 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301172    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="online"
May 25 06:57:53.301820 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301185    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="online"
May 25 06:57:53.301820 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301191    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="online"
May 25 06:57:53.301820 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301231    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="offline"
May 25 06:57:53.301820 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301240    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="online"
May 25 06:57:53.301820 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301249    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="online"
May 25 06:57:53.301820 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301255    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="ui"
May 25 06:57:53.301820 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301261    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="offline"
May 25 06:57:53.301820 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301267    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="0782bacf-2540-4526-964e-a71c4d4e8be7" containerName="registry"
May 25 06:57:53.301820 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.301273    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="a25d3e1d-f8a6-4a20-970e-05c6855a292e" containerName="ui"
May 25 06:57:53.306200 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.306177    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-fqcnr"
May 25 06:57:53.308045 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.308020    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 25 06:57:53.308045 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.308042    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
May 25 06:57:53.308313 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.308091    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 25 06:57:53.308566 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.308546    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-sptmf\""
May 25 06:57:53.319619 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.319592    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-fqcnr"]
May 25 06:57:53.377811 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.377779    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c1d67ff2-4eb1-47da-a6a0-6e49efc73005-postgresdata\") pod \"postgres-7c7656cc6d-fqcnr\" (UID: \"c1d67ff2-4eb1-47da-a6a0-6e49efc73005\") " pod="test-ns-feast/postgres-7c7656cc6d-fqcnr"
May 25 06:57:53.377984 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.377898    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xkbnx\" (UniqueName: \"kubernetes.io/projected/c1d67ff2-4eb1-47da-a6a0-6e49efc73005-kube-api-access-xkbnx\") pod \"postgres-7c7656cc6d-fqcnr\" (UID: \"c1d67ff2-4eb1-47da-a6a0-6e49efc73005\") " pod="test-ns-feast/postgres-7c7656cc6d-fqcnr"
May 25 06:57:53.479193 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.479157    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xkbnx\" (UniqueName: \"kubernetes.io/projected/c1d67ff2-4eb1-47da-a6a0-6e49efc73005-kube-api-access-xkbnx\") pod \"postgres-7c7656cc6d-fqcnr\" (UID: \"c1d67ff2-4eb1-47da-a6a0-6e49efc73005\") " pod="test-ns-feast/postgres-7c7656cc6d-fqcnr"
May 25 06:57:53.479361 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.479221    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c1d67ff2-4eb1-47da-a6a0-6e49efc73005-postgresdata\") pod \"postgres-7c7656cc6d-fqcnr\" (UID: \"c1d67ff2-4eb1-47da-a6a0-6e49efc73005\") " pod="test-ns-feast/postgres-7c7656cc6d-fqcnr"
May 25 06:57:53.479618 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.479598    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c1d67ff2-4eb1-47da-a6a0-6e49efc73005-postgresdata\") pod \"postgres-7c7656cc6d-fqcnr\" (UID: \"c1d67ff2-4eb1-47da-a6a0-6e49efc73005\") " pod="test-ns-feast/postgres-7c7656cc6d-fqcnr"
May 25 06:57:53.485848 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.485823    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xkbnx\" (UniqueName: \"kubernetes.io/projected/c1d67ff2-4eb1-47da-a6a0-6e49efc73005-kube-api-access-xkbnx\") pod \"postgres-7c7656cc6d-fqcnr\" (UID: \"c1d67ff2-4eb1-47da-a6a0-6e49efc73005\") " pod="test-ns-feast/postgres-7c7656cc6d-fqcnr"
May 25 06:57:53.630703 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.630669    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-fqcnr"
May 25 06:57:53.754203 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.754172    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-fqcnr"]
May 25 06:57:53.757294 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:57:53.757267    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc1d67ff2_4eb1_47da_a6a0_6e49efc73005.slice/crio-a7cbc7dfccdf0a2ce9e70983d4ae71cf83b36cbc22161451095fd6beabe85915 WatchSource:0}: Error finding container a7cbc7dfccdf0a2ce9e70983d4ae71cf83b36cbc22161451095fd6beabe85915: Status 404 returned error can't find the container with id a7cbc7dfccdf0a2ce9e70983d4ae71cf83b36cbc22161451095fd6beabe85915
May 25 06:57:53.759401 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:53.759383    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 06:57:54.584396 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:54.584358    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-fqcnr" event={"ID":"c1d67ff2-4eb1-47da-a6a0-6e49efc73005","Type":"ContainerStarted","Data":"a7cbc7dfccdf0a2ce9e70983d4ae71cf83b36cbc22161451095fd6beabe85915"}
May 25 06:57:59.603573 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:59.603534    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-fqcnr" event={"ID":"c1d67ff2-4eb1-47da-a6a0-6e49efc73005","Type":"ContainerStarted","Data":"fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433"}
May 25 06:57:59.617002 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:57:59.616955    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/postgres-7c7656cc6d-fqcnr" podStartSLOduration=1.321006652 podStartE2EDuration="6.61693719s" podCreationTimestamp="2026-05-25 06:57:53 +0000 UTC" firstStartedPulling="2026-05-25 06:57:53.759535512 +0000 UTC m=+711.995721881" lastFinishedPulling="2026-05-25 06:57:59.055466046 +0000 UTC m=+717.291652419" observedRunningTime="2026-05-25 06:57:59.615727786 +0000 UTC m=+717.851914179" watchObservedRunningTime="2026-05-25 06:57:59.61693719 +0000 UTC m=+717.853123583"
May 25 06:58:04.010908 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.010873    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"]
May 25 06:58:04.019228 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.019203    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.021129 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.021105    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\""
May 25 06:58:04.021129 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.021119    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\""
May 25 06:58:04.021129 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.021105    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\""
May 25 06:58:04.021339 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.021155    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 25 06:58:04.021339 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.021236    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-gb5cp\""
May 25 06:58:04.025325 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.025303    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"]
May 25 06:58:04.174290 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.174255    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-offline-tls\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.174290 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.174293    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-online-tls\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.174534 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.174314    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-trh6v\" (UniqueName: \"kubernetes.io/projected/efbf96dc-7cec-4117-b969-cfa1af521542-kube-api-access-trh6v\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.174534 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.174421    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/efbf96dc-7cec-4117-b969-cfa1af521542-feast-data\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.174534 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.174480    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-registry-tls\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.275156 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.275046    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-offline-tls\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.275156 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.275102    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-online-tls\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.275156 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.275129    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-trh6v\" (UniqueName: \"kubernetes.io/projected/efbf96dc-7cec-4117-b969-cfa1af521542-kube-api-access-trh6v\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.275427 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.275179    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/efbf96dc-7cec-4117-b969-cfa1af521542-feast-data\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.275427 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.275318    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-registry-tls\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.275609 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.275590    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/efbf96dc-7cec-4117-b969-cfa1af521542-feast-data\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.277582 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.277553    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-online-tls\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.277748 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.277727    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-registry-tls\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.277791 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.277727    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-offline-tls\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.281681 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.281654    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-trh6v\" (UniqueName: \"kubernetes.io/projected/efbf96dc-7cec-4117-b969-cfa1af521542-kube-api-access-trh6v\") pod \"feast-credit-scoring-7dbcdb945c-46fhk\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") " pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.328707 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.328665    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:04.464757 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.464622    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"]
May 25 06:58:04.467346 ip-10-0-128-6 kubenswrapper[2572]: W0525 06:58:04.467316    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podefbf96dc_7cec_4117_b969_cfa1af521542.slice/crio-14c75388cc660f75fc4d3d856e4bb8962a4b50aa233232877f89ca654da9cdf9 WatchSource:0}: Error finding container 14c75388cc660f75fc4d3d856e4bb8962a4b50aa233232877f89ca654da9cdf9: Status 404 returned error can't find the container with id 14c75388cc660f75fc4d3d856e4bb8962a4b50aa233232877f89ca654da9cdf9
May 25 06:58:04.619695 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.619655    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" event={"ID":"efbf96dc-7cec-4117-b969-cfa1af521542","Type":"ContainerStarted","Data":"a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358"}
May 25 06:58:04.619695 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:04.619700    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" event={"ID":"efbf96dc-7cec-4117-b969-cfa1af521542","Type":"ContainerStarted","Data":"14c75388cc660f75fc4d3d856e4bb8962a4b50aa233232877f89ca654da9cdf9"}
May 25 06:58:05.623646 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:05.623608    2572 generic.go:358] "Generic (PLEG): container finished" podID="efbf96dc-7cec-4117-b969-cfa1af521542" containerID="a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358" exitCode=0
May 25 06:58:05.624013 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:05.623662    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" event={"ID":"efbf96dc-7cec-4117-b969-cfa1af521542","Type":"ContainerDied","Data":"a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358"}
May 25 06:58:06.627857 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:06.627809    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" event={"ID":"efbf96dc-7cec-4117-b969-cfa1af521542","Type":"ContainerStarted","Data":"b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe"}
May 25 06:58:12.646830 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:12.646797    2572 generic.go:358] "Generic (PLEG): container finished" podID="efbf96dc-7cec-4117-b969-cfa1af521542" containerID="b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe" exitCode=0
May 25 06:58:12.647328 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:12.646873    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" event={"ID":"efbf96dc-7cec-4117-b969-cfa1af521542","Type":"ContainerDied","Data":"b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe"}
May 25 06:58:13.653550 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:13.653517    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" event={"ID":"efbf96dc-7cec-4117-b969-cfa1af521542","Type":"ContainerStarted","Data":"7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d"}
May 25 06:58:13.653550 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:13.653555    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" event={"ID":"efbf96dc-7cec-4117-b969-cfa1af521542","Type":"ContainerStarted","Data":"cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390"}
May 25 06:58:13.654211 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:13.653565    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" event={"ID":"efbf96dc-7cec-4117-b969-cfa1af521542","Type":"ContainerStarted","Data":"5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd"}
May 25 06:58:13.673057 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:13.672986    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" podStartSLOduration=10.672964212 podStartE2EDuration="10.672964212s" podCreationTimestamp="2026-05-25 06:58:03 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 06:58:13.672053854 +0000 UTC m=+731.908240247" watchObservedRunningTime="2026-05-25 06:58:13.672964212 +0000 UTC m=+731.909150606"
May 25 06:58:14.328964 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:14.328926    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:14.329146 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:14.328988    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:14.329146 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:14.329001    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:16.329870 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:16.329818    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:16.330294 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:16.330015    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:16.330294 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:16.330036    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:16.331696 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:16.331662    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused"
May 25 06:58:16.331832 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:16.331684    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.29:8816: connect: connection refused"
May 25 06:58:16.331832 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:16.331662    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 25 06:58:19.329533 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:19.329481    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused"
May 25 06:58:19.330491 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:19.330467    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:19.331391 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:19.331371    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:19.334766 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:19.334744    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:19.679132 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:19.679046    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:22.329999 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:22.329951    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused"
May 25 06:58:25.330175 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:25.330142    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:58:25.330891 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:58:25.330872    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:59:26.983731 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:26.983651    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"]
May 25 06:59:26.984312 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:26.983989    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="registry" containerID="cri-o://5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd" gracePeriod=30
May 25 06:59:26.984312 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:26.984044    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="offline" containerID="cri-o://7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d" gracePeriod=30
May 25 06:59:26.984312 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:26.984074    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="online" containerID="cri-o://cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390" gracePeriod=30
May 25 06:59:26.999684 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:26.999653    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-fqcnr"]
May 25 06:59:26.999879 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:26.999856    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/postgres-7c7656cc6d-fqcnr" podUID="c1d67ff2-4eb1-47da-a6a0-6e49efc73005" containerName="postgres" containerID="cri-o://fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433" gracePeriod=30
May 25 06:59:27.238078 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.238009    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-fqcnr"
May 25 06:59:27.300092 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.300057    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xkbnx\" (UniqueName: \"kubernetes.io/projected/c1d67ff2-4eb1-47da-a6a0-6e49efc73005-kube-api-access-xkbnx\") pod \"c1d67ff2-4eb1-47da-a6a0-6e49efc73005\" (UID: \"c1d67ff2-4eb1-47da-a6a0-6e49efc73005\") "
May 25 06:59:27.300250 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.300171    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c1d67ff2-4eb1-47da-a6a0-6e49efc73005-postgresdata\") pod \"c1d67ff2-4eb1-47da-a6a0-6e49efc73005\" (UID: \"c1d67ff2-4eb1-47da-a6a0-6e49efc73005\") "
May 25 06:59:27.302190 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.302161    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c1d67ff2-4eb1-47da-a6a0-6e49efc73005-kube-api-access-xkbnx" (OuterVolumeSpecName: "kube-api-access-xkbnx") pod "c1d67ff2-4eb1-47da-a6a0-6e49efc73005" (UID: "c1d67ff2-4eb1-47da-a6a0-6e49efc73005"). InnerVolumeSpecName "kube-api-access-xkbnx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 06:59:27.332814 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.332763    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c1d67ff2-4eb1-47da-a6a0-6e49efc73005-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "c1d67ff2-4eb1-47da-a6a0-6e49efc73005" (UID: "c1d67ff2-4eb1-47da-a6a0-6e49efc73005"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 06:59:27.401308 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.401268    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xkbnx\" (UniqueName: \"kubernetes.io/projected/c1d67ff2-4eb1-47da-a6a0-6e49efc73005-kube-api-access-xkbnx\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:59:27.401308 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.401309    2572 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c1d67ff2-4eb1-47da-a6a0-6e49efc73005-postgresdata\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:59:27.881251 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.881216    2572 generic.go:358] "Generic (PLEG): container finished" podID="c1d67ff2-4eb1-47da-a6a0-6e49efc73005" containerID="fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433" exitCode=0
May 25 06:59:27.881432 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.881274    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-fqcnr"
May 25 06:59:27.881432 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.881308    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-fqcnr" event={"ID":"c1d67ff2-4eb1-47da-a6a0-6e49efc73005","Type":"ContainerDied","Data":"fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433"}
May 25 06:59:27.881432 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.881352    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-fqcnr" event={"ID":"c1d67ff2-4eb1-47da-a6a0-6e49efc73005","Type":"ContainerDied","Data":"a7cbc7dfccdf0a2ce9e70983d4ae71cf83b36cbc22161451095fd6beabe85915"}
May 25 06:59:27.881432 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.881375    2572 scope.go:117] "RemoveContainer" containerID="fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433"
May 25 06:59:27.883989 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.883963    2572 generic.go:358] "Generic (PLEG): container finished" podID="efbf96dc-7cec-4117-b969-cfa1af521542" containerID="cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390" exitCode=0
May 25 06:59:27.884108 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.884039    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" event={"ID":"efbf96dc-7cec-4117-b969-cfa1af521542","Type":"ContainerDied","Data":"cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390"}
May 25 06:59:27.889425 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.889408    2572 scope.go:117] "RemoveContainer" containerID="fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433"
May 25 06:59:27.889694 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:59:27.889675    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433\": container with ID starting with fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433 not found: ID does not exist" containerID="fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433"
May 25 06:59:27.889749 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.889705    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433"} err="failed to get container status \"fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433\": rpc error: code = NotFound desc = could not find container \"fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433\": container with ID starting with fedc6db23a11c207d34229e81e08260b8a9ef855cc60384aef325977edd58433 not found: ID does not exist"
May 25 06:59:27.900611 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.900553    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-fqcnr"]
May 25 06:59:27.901726 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:27.901708    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-fqcnr"]
May 25 06:59:28.282267 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:28.282233    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c1d67ff2-4eb1-47da-a6a0-6e49efc73005" path="/var/lib/kubelet/pods/c1d67ff2-4eb1-47da-a6a0-6e49efc73005/volumes"
May 25 06:59:29.675137 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:29.675095    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 25 06:59:39.675342 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:39.675299    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 25 06:59:49.675619 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:49.675557    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 25 06:59:49.676019 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:49.675709    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:59:57.631566 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.631541    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:59:57.746538 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.746490    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/efbf96dc-7cec-4117-b969-cfa1af521542-feast-data\") pod \"efbf96dc-7cec-4117-b969-cfa1af521542\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") "
May 25 06:59:57.746706 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.746576    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-registry-tls\") pod \"efbf96dc-7cec-4117-b969-cfa1af521542\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") "
May 25 06:59:57.746706 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.746626    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-online-tls\") pod \"efbf96dc-7cec-4117-b969-cfa1af521542\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") "
May 25 06:59:57.746706 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.746677    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-offline-tls\") pod \"efbf96dc-7cec-4117-b969-cfa1af521542\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") "
May 25 06:59:57.746884 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.746706    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-trh6v\" (UniqueName: \"kubernetes.io/projected/efbf96dc-7cec-4117-b969-cfa1af521542-kube-api-access-trh6v\") pod \"efbf96dc-7cec-4117-b969-cfa1af521542\" (UID: \"efbf96dc-7cec-4117-b969-cfa1af521542\") "
May 25 06:59:57.748912 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.748816    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/efbf96dc-7cec-4117-b969-cfa1af521542-kube-api-access-trh6v" (OuterVolumeSpecName: "kube-api-access-trh6v") pod "efbf96dc-7cec-4117-b969-cfa1af521542" (UID: "efbf96dc-7cec-4117-b969-cfa1af521542"). InnerVolumeSpecName "kube-api-access-trh6v". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 06:59:57.748912 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.748831    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-online-tls" (OuterVolumeSpecName: "online-tls") pod "efbf96dc-7cec-4117-b969-cfa1af521542" (UID: "efbf96dc-7cec-4117-b969-cfa1af521542"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:59:57.748912 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.748888    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "efbf96dc-7cec-4117-b969-cfa1af521542" (UID: "efbf96dc-7cec-4117-b969-cfa1af521542"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:59:57.749202 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.749180    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "efbf96dc-7cec-4117-b969-cfa1af521542" (UID: "efbf96dc-7cec-4117-b969-cfa1af521542"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 06:59:57.758994 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.758966    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/efbf96dc-7cec-4117-b969-cfa1af521542-feast-data" (OuterVolumeSpecName: "feast-data") pod "efbf96dc-7cec-4117-b969-cfa1af521542" (UID: "efbf96dc-7cec-4117-b969-cfa1af521542"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 06:59:57.847399 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.847362    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-trh6v\" (UniqueName: \"kubernetes.io/projected/efbf96dc-7cec-4117-b969-cfa1af521542-kube-api-access-trh6v\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:59:57.847399 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.847395    2572 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/efbf96dc-7cec-4117-b969-cfa1af521542-feast-data\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:59:57.847557 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.847409    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:59:57.847557 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.847423    2572 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-online-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:59:57.847557 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.847433    2572 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/efbf96dc-7cec-4117-b969-cfa1af521542-offline-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 06:59:57.977647 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.977615    2572 generic.go:358] "Generic (PLEG): container finished" podID="efbf96dc-7cec-4117-b969-cfa1af521542" containerID="7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d" exitCode=137
May 25 06:59:57.977647 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.977645    2572 generic.go:358] "Generic (PLEG): container finished" podID="efbf96dc-7cec-4117-b969-cfa1af521542" containerID="5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd" exitCode=137
May 25 06:59:57.977835 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.977685    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" event={"ID":"efbf96dc-7cec-4117-b969-cfa1af521542","Type":"ContainerDied","Data":"7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d"}
May 25 06:59:57.977835 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.977718    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"
May 25 06:59:57.977835 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.977734    2572 scope.go:117] "RemoveContainer" containerID="7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d"
May 25 06:59:57.977961 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.977722    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" event={"ID":"efbf96dc-7cec-4117-b969-cfa1af521542","Type":"ContainerDied","Data":"5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd"}
May 25 06:59:57.977961 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.977871    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk" event={"ID":"efbf96dc-7cec-4117-b969-cfa1af521542","Type":"ContainerDied","Data":"14c75388cc660f75fc4d3d856e4bb8962a4b50aa233232877f89ca654da9cdf9"}
May 25 06:59:57.985969 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.985951    2572 scope.go:117] "RemoveContainer" containerID="cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390"
May 25 06:59:57.995548 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.995525    2572 scope.go:117] "RemoveContainer" containerID="5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd"
May 25 06:59:57.997723 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:57.997699    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"]
May 25 06:59:58.000693 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.000675    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-7dbcdb945c-46fhk"]
May 25 06:59:58.003831 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.003814    2572 scope.go:117] "RemoveContainer" containerID="b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe"
May 25 06:59:58.010548 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.010525    2572 scope.go:117] "RemoveContainer" containerID="a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358"
May 25 06:59:58.017079 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.017055    2572 scope.go:117] "RemoveContainer" containerID="7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d"
May 25 06:59:58.017340 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:59:58.017320    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d\": container with ID starting with 7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d not found: ID does not exist" containerID="7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d"
May 25 06:59:58.017394 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.017349    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d"} err="failed to get container status \"7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d\": rpc error: code = NotFound desc = could not find container \"7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d\": container with ID starting with 7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d not found: ID does not exist"
May 25 06:59:58.017394 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.017368    2572 scope.go:117] "RemoveContainer" containerID="cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390"
May 25 06:59:58.017597 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:59:58.017578    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390\": container with ID starting with cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390 not found: ID does not exist" containerID="cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390"
May 25 06:59:58.017659 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.017607    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390"} err="failed to get container status \"cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390\": rpc error: code = NotFound desc = could not find container \"cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390\": container with ID starting with cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390 not found: ID does not exist"
May 25 06:59:58.017659 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.017632    2572 scope.go:117] "RemoveContainer" containerID="5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd"
May 25 06:59:58.017886 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:59:58.017865    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd\": container with ID starting with 5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd not found: ID does not exist" containerID="5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd"
May 25 06:59:58.017934 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.017895    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd"} err="failed to get container status \"5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd\": rpc error: code = NotFound desc = could not find container \"5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd\": container with ID starting with 5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd not found: ID does not exist"
May 25 06:59:58.017934 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.017912    2572 scope.go:117] "RemoveContainer" containerID="b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe"
May 25 06:59:58.018104 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:59:58.018089    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe\": container with ID starting with b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe not found: ID does not exist" containerID="b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe"
May 25 06:59:58.018146 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.018107    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe"} err="failed to get container status \"b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe\": rpc error: code = NotFound desc = could not find container \"b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe\": container with ID starting with b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe not found: ID does not exist"
May 25 06:59:58.018146 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.018121    2572 scope.go:117] "RemoveContainer" containerID="a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358"
May 25 06:59:58.018307 ip-10-0-128-6 kubenswrapper[2572]: E0525 06:59:58.018292    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358\": container with ID starting with a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358 not found: ID does not exist" containerID="a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358"
May 25 06:59:58.018343 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.018309    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358"} err="failed to get container status \"a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358\": rpc error: code = NotFound desc = could not find container \"a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358\": container with ID starting with a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358 not found: ID does not exist"
May 25 06:59:58.018343 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.018322    2572 scope.go:117] "RemoveContainer" containerID="7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d"
May 25 06:59:58.018524 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.018495    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d"} err="failed to get container status \"7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d\": rpc error: code = NotFound desc = could not find container \"7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d\": container with ID starting with 7df067fc76a3aa94c0c7de2414ae5dd0da669f33022d3315ad2740997761c74d not found: ID does not exist"
May 25 06:59:58.018592 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.018527    2572 scope.go:117] "RemoveContainer" containerID="cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390"
May 25 06:59:58.018710 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.018694    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390"} err="failed to get container status \"cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390\": rpc error: code = NotFound desc = could not find container \"cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390\": container with ID starting with cdaef35b1808a296062a041a868ec9daf8ce9d957ebed64a9369496b3a041390 not found: ID does not exist"
May 25 06:59:58.018753 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.018710    2572 scope.go:117] "RemoveContainer" containerID="5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd"
May 25 06:59:58.018943 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.018921    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd"} err="failed to get container status \"5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd\": rpc error: code = NotFound desc = could not find container \"5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd\": container with ID starting with 5beffc1427526ec46ebd9da1099b412d7dff10226f12224ff27fd023fa6307dd not found: ID does not exist"
May 25 06:59:58.018943 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.018942    2572 scope.go:117] "RemoveContainer" containerID="b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe"
May 25 06:59:58.019162 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.019144    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe"} err="failed to get container status \"b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe\": rpc error: code = NotFound desc = could not find container \"b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe\": container with ID starting with b051f446e4680de1aef6e39ccd1f1720e199111c3125e9125ea688b61fd0aebe not found: ID does not exist"
May 25 06:59:58.019209 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.019165    2572 scope.go:117] "RemoveContainer" containerID="a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358"
May 25 06:59:58.019354 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.019336    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358"} err="failed to get container status \"a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358\": rpc error: code = NotFound desc = could not find container \"a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358\": container with ID starting with a6eb5f37bd6d1c345c76c733b5824b4ce8f4aa68edbb949796c2e77ac0eaf358 not found: ID does not exist"
May 25 06:59:58.282534 ip-10-0-128-6 kubenswrapper[2572]: I0525 06:59:58.282453    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" path="/var/lib/kubelet/pods/efbf96dc-7cec-4117-b969-cfa1af521542/volumes"
May 25 07:01:02.260917 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:02.260828    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 07:01:02.263484 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:02.263464    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 07:01:36.821541 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821503    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-854544c579-wpcpw"]
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821794    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="online"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821805    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="online"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821815    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c1d67ff2-4eb1-47da-a6a0-6e49efc73005" containerName="postgres"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821821    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="c1d67ff2-4eb1-47da-a6a0-6e49efc73005" containerName="postgres"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821830    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="feast-apply"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821836    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="feast-apply"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821856    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="registry"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821861    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="registry"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821871    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="offline"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821876    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="offline"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821885    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="feast-init"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821890    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="feast-init"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821933    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="online"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821942    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="c1d67ff2-4eb1-47da-a6a0-6e49efc73005" containerName="postgres"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821948    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="registry"
May 25 07:01:36.822060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.821954    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="efbf96dc-7cec-4117-b969-cfa1af521542" containerName="offline"
May 25 07:01:36.824707 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.824680    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-wpcpw"
May 25 07:01:36.826896 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.826870    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
May 25 07:01:36.827040 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.826906    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
May 25 07:01:36.827407 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.827387    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-t2w9z\""
May 25 07:01:36.840789 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.840754    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-wpcpw"]
May 25 07:01:36.923572 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:36.923525    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4gtk\" (UniqueName: \"kubernetes.io/projected/4d617185-b2c6-45bd-b1f8-bd6578d767bf-kube-api-access-w4gtk\") pod \"redis-854544c579-wpcpw\" (UID: \"4d617185-b2c6-45bd-b1f8-bd6578d767bf\") " pod="test-ns-feast-rest/redis-854544c579-wpcpw"
May 25 07:01:37.024502 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:37.024467    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w4gtk\" (UniqueName: \"kubernetes.io/projected/4d617185-b2c6-45bd-b1f8-bd6578d767bf-kube-api-access-w4gtk\") pod \"redis-854544c579-wpcpw\" (UID: \"4d617185-b2c6-45bd-b1f8-bd6578d767bf\") " pod="test-ns-feast-rest/redis-854544c579-wpcpw"
May 25 07:01:37.032371 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:37.032337    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4gtk\" (UniqueName: \"kubernetes.io/projected/4d617185-b2c6-45bd-b1f8-bd6578d767bf-kube-api-access-w4gtk\") pod \"redis-854544c579-wpcpw\" (UID: \"4d617185-b2c6-45bd-b1f8-bd6578d767bf\") " pod="test-ns-feast-rest/redis-854544c579-wpcpw"
May 25 07:01:37.146820 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:37.146727    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-wpcpw"
May 25 07:01:37.273509 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:37.273477    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-wpcpw"]
May 25 07:01:37.276523 ip-10-0-128-6 kubenswrapper[2572]: W0525 07:01:37.276496    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod4d617185_b2c6_45bd_b1f8_bd6578d767bf.slice/crio-4ac7f2587604d0f8f0afea030cfdf925795eae462237661b3ad7c321fad03096 WatchSource:0}: Error finding container 4ac7f2587604d0f8f0afea030cfdf925795eae462237661b3ad7c321fad03096: Status 404 returned error can't find the container with id 4ac7f2587604d0f8f0afea030cfdf925795eae462237661b3ad7c321fad03096
May 25 07:01:38.264104 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:38.264057    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-wpcpw" event={"ID":"4d617185-b2c6-45bd-b1f8-bd6578d767bf","Type":"ContainerStarted","Data":"4ac7f2587604d0f8f0afea030cfdf925795eae462237661b3ad7c321fad03096"}
May 25 07:01:39.268654 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:39.268560    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-wpcpw" event={"ID":"4d617185-b2c6-45bd-b1f8-bd6578d767bf","Type":"ContainerStarted","Data":"79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318"}
May 25 07:01:39.280993 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:39.280949    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/redis-854544c579-wpcpw" podStartSLOduration=1.729801855 podStartE2EDuration="3.280932926s" podCreationTimestamp="2026-05-25 07:01:36 +0000 UTC" firstStartedPulling="2026-05-25 07:01:37.278149966 +0000 UTC m=+935.514336349" lastFinishedPulling="2026-05-25 07:01:38.829281051 +0000 UTC m=+937.065467420" observedRunningTime="2026-05-25 07:01:39.280574055 +0000 UTC m=+937.516760449" watchObservedRunningTime="2026-05-25 07:01:39.280932926 +0000 UTC m=+937.517119318"
May 25 07:01:42.266089 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:42.266055    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"]
May 25 07:01:42.269217 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:42.269201    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"
May 25 07:01:42.271045 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:42.271022    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\""
May 25 07:01:42.275376 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:42.275355    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"]
May 25 07:01:42.365960 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:42.365919    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/6eb5de76-17d3-49e8-940d-5a96e8c6e849-postgresdata\") pod \"postgres-7c7656cc6d-vnqxr\" (UID: \"6eb5de76-17d3-49e8-940d-5a96e8c6e849\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"
May 25 07:01:42.366153 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:42.366077    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pvv68\" (UniqueName: \"kubernetes.io/projected/6eb5de76-17d3-49e8-940d-5a96e8c6e849-kube-api-access-pvv68\") pod \"postgres-7c7656cc6d-vnqxr\" (UID: \"6eb5de76-17d3-49e8-940d-5a96e8c6e849\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"
May 25 07:01:42.467303 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:42.467259    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/6eb5de76-17d3-49e8-940d-5a96e8c6e849-postgresdata\") pod \"postgres-7c7656cc6d-vnqxr\" (UID: \"6eb5de76-17d3-49e8-940d-5a96e8c6e849\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"
May 25 07:01:42.467479 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:42.467342    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pvv68\" (UniqueName: \"kubernetes.io/projected/6eb5de76-17d3-49e8-940d-5a96e8c6e849-kube-api-access-pvv68\") pod \"postgres-7c7656cc6d-vnqxr\" (UID: \"6eb5de76-17d3-49e8-940d-5a96e8c6e849\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"
May 25 07:01:42.467656 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:42.467632    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/6eb5de76-17d3-49e8-940d-5a96e8c6e849-postgresdata\") pod \"postgres-7c7656cc6d-vnqxr\" (UID: \"6eb5de76-17d3-49e8-940d-5a96e8c6e849\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"
May 25 07:01:42.474316 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:42.474292    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pvv68\" (UniqueName: \"kubernetes.io/projected/6eb5de76-17d3-49e8-940d-5a96e8c6e849-kube-api-access-pvv68\") pod \"postgres-7c7656cc6d-vnqxr\" (UID: \"6eb5de76-17d3-49e8-940d-5a96e8c6e849\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"
May 25 07:01:42.580998 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:42.580959    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"
May 25 07:01:42.705923 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:42.705751    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"]
May 25 07:01:42.708436 ip-10-0-128-6 kubenswrapper[2572]: W0525 07:01:42.708402    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6eb5de76_17d3_49e8_940d_5a96e8c6e849.slice/crio-62cbeab672084c8ce0fa341208551dc5e7cd5bf88d0da618c3e883020853c5b5 WatchSource:0}: Error finding container 62cbeab672084c8ce0fa341208551dc5e7cd5bf88d0da618c3e883020853c5b5: Status 404 returned error can't find the container with id 62cbeab672084c8ce0fa341208551dc5e7cd5bf88d0da618c3e883020853c5b5
May 25 07:01:43.282237 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:43.282207    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr" event={"ID":"6eb5de76-17d3-49e8-940d-5a96e8c6e849","Type":"ContainerStarted","Data":"c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7"}
May 25 07:01:43.282237 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:43.282239    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr" event={"ID":"6eb5de76-17d3-49e8-940d-5a96e8c6e849","Type":"ContainerStarted","Data":"62cbeab672084c8ce0fa341208551dc5e7cd5bf88d0da618c3e883020853c5b5"}
May 25 07:01:43.295169 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:43.295115    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr" podStartSLOduration=1.295099849 podStartE2EDuration="1.295099849s" podCreationTimestamp="2026-05-25 07:01:42 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 07:01:43.293576126 +0000 UTC m=+941.529762513" watchObservedRunningTime="2026-05-25 07:01:43.295099849 +0000 UTC m=+941.531286265"
May 25 07:01:47.912585 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:47.912549    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"]
May 25 07:01:47.918830 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:47.918811    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:47.920944 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:47.920924    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\""
May 25 07:01:47.920944 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:47.920939    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
May 25 07:01:47.921132 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:47.920990    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-rhl8g\""
May 25 07:01:47.921132 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:47.920991    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\""
May 25 07:01:47.926086 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:47.926036    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"]
May 25 07:01:48.012002 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.011950    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/aa90d273-1d8b-4839-9c9b-1fe7241526bb-online-tls\") pod \"feast-credit-scoring-6bcfb77767-6hbzw\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") " pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.012002 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.011998    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/aa90d273-1d8b-4839-9c9b-1fe7241526bb-feast-data\") pod \"feast-credit-scoring-6bcfb77767-6hbzw\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") " pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.012224 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.012100    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xh89w\" (UniqueName: \"kubernetes.io/projected/aa90d273-1d8b-4839-9c9b-1fe7241526bb-kube-api-access-xh89w\") pod \"feast-credit-scoring-6bcfb77767-6hbzw\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") " pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.012224 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.012137    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/aa90d273-1d8b-4839-9c9b-1fe7241526bb-registry-tls\") pod \"feast-credit-scoring-6bcfb77767-6hbzw\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") " pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.113499 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.113463    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/aa90d273-1d8b-4839-9c9b-1fe7241526bb-feast-data\") pod \"feast-credit-scoring-6bcfb77767-6hbzw\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") " pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.113654 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.113524    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xh89w\" (UniqueName: \"kubernetes.io/projected/aa90d273-1d8b-4839-9c9b-1fe7241526bb-kube-api-access-xh89w\") pod \"feast-credit-scoring-6bcfb77767-6hbzw\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") " pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.113654 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.113549    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/aa90d273-1d8b-4839-9c9b-1fe7241526bb-registry-tls\") pod \"feast-credit-scoring-6bcfb77767-6hbzw\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") " pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.113654 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.113584    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/aa90d273-1d8b-4839-9c9b-1fe7241526bb-online-tls\") pod \"feast-credit-scoring-6bcfb77767-6hbzw\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") " pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.113900 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.113876    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/aa90d273-1d8b-4839-9c9b-1fe7241526bb-feast-data\") pod \"feast-credit-scoring-6bcfb77767-6hbzw\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") " pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.116257 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.116225    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/aa90d273-1d8b-4839-9c9b-1fe7241526bb-registry-tls\") pod \"feast-credit-scoring-6bcfb77767-6hbzw\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") " pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.116257 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.116240    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/aa90d273-1d8b-4839-9c9b-1fe7241526bb-online-tls\") pod \"feast-credit-scoring-6bcfb77767-6hbzw\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") " pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.121279 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.121257    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xh89w\" (UniqueName: \"kubernetes.io/projected/aa90d273-1d8b-4839-9c9b-1fe7241526bb-kube-api-access-xh89w\") pod \"feast-credit-scoring-6bcfb77767-6hbzw\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") " pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.228263 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.228179    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:01:48.362071 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:48.362039    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"]
May 25 07:01:48.365074 ip-10-0-128-6 kubenswrapper[2572]: W0525 07:01:48.365052    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podaa90d273_1d8b_4839_9c9b_1fe7241526bb.slice/crio-aba1dcc9fe1e7cd64cb89c8c8d273ba69f6a8e16dcc2903a99acf5b1844b39a9 WatchSource:0}: Error finding container aba1dcc9fe1e7cd64cb89c8c8d273ba69f6a8e16dcc2903a99acf5b1844b39a9: Status 404 returned error can't find the container with id aba1dcc9fe1e7cd64cb89c8c8d273ba69f6a8e16dcc2903a99acf5b1844b39a9
May 25 07:01:49.299604 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:49.299569    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" event={"ID":"aa90d273-1d8b-4839-9c9b-1fe7241526bb","Type":"ContainerStarted","Data":"b960325924d20cc190cbf5e42acbb796a694b89affb93d472c974865da86e2c2"}
May 25 07:01:49.299604 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:49.299606    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" event={"ID":"aa90d273-1d8b-4839-9c9b-1fe7241526bb","Type":"ContainerStarted","Data":"aba1dcc9fe1e7cd64cb89c8c8d273ba69f6a8e16dcc2903a99acf5b1844b39a9"}
May 25 07:01:50.304047 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:50.304004    2572 generic.go:358] "Generic (PLEG): container finished" podID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerID="b960325924d20cc190cbf5e42acbb796a694b89affb93d472c974865da86e2c2" exitCode=0
May 25 07:01:50.304524 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:50.304089    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" event={"ID":"aa90d273-1d8b-4839-9c9b-1fe7241526bb","Type":"ContainerDied","Data":"b960325924d20cc190cbf5e42acbb796a694b89affb93d472c974865da86e2c2"}
May 25 07:01:51.308716 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:51.308679    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" event={"ID":"aa90d273-1d8b-4839-9c9b-1fe7241526bb","Type":"ContainerStarted","Data":"211c66cf6d99f0b4b9c53fd957ee72108923098ea2d4dc23f4dbe5858362e7b3"}
May 25 07:01:58.340007 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:58.339971    2572 generic.go:358] "Generic (PLEG): container finished" podID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerID="211c66cf6d99f0b4b9c53fd957ee72108923098ea2d4dc23f4dbe5858362e7b3" exitCode=0
May 25 07:01:58.340425 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:58.340038    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" event={"ID":"aa90d273-1d8b-4839-9c9b-1fe7241526bb","Type":"ContainerDied","Data":"211c66cf6d99f0b4b9c53fd957ee72108923098ea2d4dc23f4dbe5858362e7b3"}
May 25 07:01:59.345660 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:59.345628    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" event={"ID":"aa90d273-1d8b-4839-9c9b-1fe7241526bb","Type":"ContainerStarted","Data":"10b1d1f5485ed0cb392530857264c6c34f5bd378f64d32bdaa1e162bf1234479"}
May 25 07:01:59.346038 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:59.345671    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" event={"ID":"aa90d273-1d8b-4839-9c9b-1fe7241526bb","Type":"ContainerStarted","Data":"a849b4e99a2c61a68654e351f3b0c23a37692d61079af4f1da5cdecc1c0e8945"}
May 25 07:01:59.364131 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:01:59.364073    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" podStartSLOduration=12.36405582 podStartE2EDuration="12.36405582s" podCreationTimestamp="2026-05-25 07:01:47 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 07:01:59.362385676 +0000 UTC m=+957.598572069" watchObservedRunningTime="2026-05-25 07:01:59.36405582 +0000 UTC m=+957.600242213"
May 25 07:02:00.228592 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:00.228553    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:02:00.228592 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:00.228600    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:02:00.230212 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:00.230171    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 25 07:02:00.230345 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:00.230226    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 25 07:02:03.229329 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:03.229282    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 25 07:02:03.229329 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:03.229315    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 25 07:02:06.229321 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:06.229273    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 25 07:02:06.234594 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:06.234561    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:02:06.235098 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:06.235075    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:02:06.237624 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:06.237600    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:02:08.228956 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:08.228915    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:02:09.229405 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:09.229365    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 25 07:02:12.228927 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:12.228895    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:02:12.229591 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:12.229570    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:02:13.713730 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.713690    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"]
May 25 07:02:13.716793 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.716773    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:13.718823 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.718801    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\""
May 25 07:02:13.718954 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.718862    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-5v9mt\""
May 25 07:02:13.719248 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.719207    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\""
May 25 07:02:13.719485 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.719468    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
May 25 07:02:13.729391 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.725934    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"]
May 25 07:02:13.837775 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.837742    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-online-tls\") pod \"feast-driver-ranking-54f4667468-wwpvg\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") " pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:13.837967 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.837877    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-registry-tls\") pod \"feast-driver-ranking-54f4667468-wwpvg\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") " pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:13.837967 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.837908    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-glvv2\" (UniqueName: \"kubernetes.io/projected/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-kube-api-access-glvv2\") pod \"feast-driver-ranking-54f4667468-wwpvg\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") " pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:13.837967 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.837927    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-feast-data\") pod \"feast-driver-ranking-54f4667468-wwpvg\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") " pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:13.938935 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.938894    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-registry-tls\") pod \"feast-driver-ranking-54f4667468-wwpvg\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") " pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:13.939133 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.938942    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-glvv2\" (UniqueName: \"kubernetes.io/projected/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-kube-api-access-glvv2\") pod \"feast-driver-ranking-54f4667468-wwpvg\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") " pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:13.939133 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.939055    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-feast-data\") pod \"feast-driver-ranking-54f4667468-wwpvg\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") " pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:13.939133 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.939105    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-online-tls\") pod \"feast-driver-ranking-54f4667468-wwpvg\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") " pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:13.939490 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.939463    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-feast-data\") pod \"feast-driver-ranking-54f4667468-wwpvg\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") " pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:13.941539 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.941519    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-online-tls\") pod \"feast-driver-ranking-54f4667468-wwpvg\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") " pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:13.941628 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.941567    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-registry-tls\") pod \"feast-driver-ranking-54f4667468-wwpvg\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") " pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:13.945708 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:13.945679    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-glvv2\" (UniqueName: \"kubernetes.io/projected/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-kube-api-access-glvv2\") pod \"feast-driver-ranking-54f4667468-wwpvg\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") " pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:14.032377 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:14.032298    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:14.165001 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:14.164831    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"]
May 25 07:02:14.168003 ip-10-0-128-6 kubenswrapper[2572]: W0525 07:02:14.167975    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5022b9a_d40e_48e5_a0e5_a5f088d0a81b.slice/crio-3f5602092409f7025af849f08065301ba2c841a91806bd8c8ac65a1317812e0a WatchSource:0}: Error finding container 3f5602092409f7025af849f08065301ba2c841a91806bd8c8ac65a1317812e0a: Status 404 returned error can't find the container with id 3f5602092409f7025af849f08065301ba2c841a91806bd8c8ac65a1317812e0a
May 25 07:02:14.397858 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:14.389153    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" event={"ID":"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b","Type":"ContainerStarted","Data":"abeeb8344f056a779cf8c6dc300824541b169de85d34564221dd6f41ccb47445"}
May 25 07:02:14.397858 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:14.389198    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" event={"ID":"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b","Type":"ContainerStarted","Data":"3f5602092409f7025af849f08065301ba2c841a91806bd8c8ac65a1317812e0a"}
May 25 07:02:19.406737 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:19.406705    2572 generic.go:358] "Generic (PLEG): container finished" podID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerID="abeeb8344f056a779cf8c6dc300824541b169de85d34564221dd6f41ccb47445" exitCode=0
May 25 07:02:19.407105 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:19.406780    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" event={"ID":"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b","Type":"ContainerDied","Data":"abeeb8344f056a779cf8c6dc300824541b169de85d34564221dd6f41ccb47445"}
May 25 07:02:20.411379 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:20.411341    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" event={"ID":"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b","Type":"ContainerStarted","Data":"fb703e24d51b88749db5727eb8c2ff5e8516cf91e973375aa354e723186ede54"}
May 25 07:02:27.433377 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:27.433345    2572 generic.go:358] "Generic (PLEG): container finished" podID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerID="fb703e24d51b88749db5727eb8c2ff5e8516cf91e973375aa354e723186ede54" exitCode=0
May 25 07:02:27.433837 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:27.433422    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" event={"ID":"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b","Type":"ContainerDied","Data":"fb703e24d51b88749db5727eb8c2ff5e8516cf91e973375aa354e723186ede54"}
May 25 07:02:28.438607 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:28.438562    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" event={"ID":"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b","Type":"ContainerStarted","Data":"b3ad88a3d8569ef43ba663607d165b04affa8ccb57e14311c371f2a4ed9fb20d"}
May 25 07:02:28.438985 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:28.438620    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" event={"ID":"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b","Type":"ContainerStarted","Data":"3c1f15894aa689562ff023aee002fa9cfa652e8c78a0464df7c61c2f57fb6467"}
May 25 07:02:28.460906 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:28.460832    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" podStartSLOduration=15.460812813 podStartE2EDuration="15.460812813s" podCreationTimestamp="2026-05-25 07:02:13 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 07:02:28.458557918 +0000 UTC m=+986.694744314" watchObservedRunningTime="2026-05-25 07:02:28.460812813 +0000 UTC m=+986.696999206"
May 25 07:02:29.033525 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:29.033484    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:29.033525 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:29.033530    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:29.035108 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:29.035068    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused"
May 25 07:02:29.035248 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:29.035104    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 25 07:02:32.033379 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:32.033335    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused"
May 25 07:02:32.033750 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:32.033334    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 25 07:02:34.033091 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:34.033052    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:34.033091 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:34.033103    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:35.032737 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:35.032688    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused"
May 25 07:02:35.038144 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:35.038114    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:35.041499 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:35.041472    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:38.033369 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:38.033323    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused"
May 25 07:02:41.033096 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:41.033055    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:02:41.033823 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:02:41.033802    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:03:58.454354 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.454279    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"]
May 25 07:03:58.454900 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.454510    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp" podUID="b4286131-2303-4bed-8c12-7945370edbfa" containerName="manager" containerID="cri-o://9a3a01db961e50b1b443da2013fd98e6cfa79d77ba35732f0edec14e7a5d07f0" gracePeriod=10
May 25 07:03:58.663868 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.663368    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"]
May 25 07:03:58.663868 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.663778    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="registry" containerID="cri-o://a849b4e99a2c61a68654e351f3b0c23a37692d61079af4f1da5cdecc1c0e8945" gracePeriod=30
May 25 07:03:58.664155 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.663861    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="online" containerID="cri-o://10b1d1f5485ed0cb392530857264c6c34f5bd378f64d32bdaa1e162bf1234479" gracePeriod=30
May 25 07:03:58.665911 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.665835    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"]
May 25 07:03:58.717257 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.717170    2572 generic.go:358] "Generic (PLEG): container finished" podID="b4286131-2303-4bed-8c12-7945370edbfa" containerID="9a3a01db961e50b1b443da2013fd98e6cfa79d77ba35732f0edec14e7a5d07f0" exitCode=0
May 25 07:03:58.717257 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.717241    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp" event={"ID":"b4286131-2303-4bed-8c12-7945370edbfa","Type":"ContainerDied","Data":"9a3a01db961e50b1b443da2013fd98e6cfa79d77ba35732f0edec14e7a5d07f0"}
May 25 07:03:58.717500 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.717472    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="registry" containerID="cri-o://3c1f15894aa689562ff023aee002fa9cfa652e8c78a0464df7c61c2f57fb6467" gracePeriod=30
May 25 07:03:58.717671 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.717614    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="online" containerID="cri-o://b3ad88a3d8569ef43ba663607d165b04affa8ccb57e14311c371f2a4ed9fb20d" gracePeriod=30
May 25 07:03:58.734730 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.734709    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"
May 25 07:03:58.831415 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.831382    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f8z7j\" (UniqueName: \"kubernetes.io/projected/b4286131-2303-4bed-8c12-7945370edbfa-kube-api-access-f8z7j\") pod \"b4286131-2303-4bed-8c12-7945370edbfa\" (UID: \"b4286131-2303-4bed-8c12-7945370edbfa\") "
May 25 07:03:58.833493 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.833469    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/b4286131-2303-4bed-8c12-7945370edbfa-kube-api-access-f8z7j" (OuterVolumeSpecName: "kube-api-access-f8z7j") pod "b4286131-2303-4bed-8c12-7945370edbfa" (UID: "b4286131-2303-4bed-8c12-7945370edbfa"). InnerVolumeSpecName "kube-api-access-f8z7j". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 07:03:58.932081 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:58.932039    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f8z7j\" (UniqueName: \"kubernetes.io/projected/b4286131-2303-4bed-8c12-7945370edbfa-kube-api-access-f8z7j\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:03:59.722793 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:59.722714    2572 generic.go:358] "Generic (PLEG): container finished" podID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerID="b3ad88a3d8569ef43ba663607d165b04affa8ccb57e14311c371f2a4ed9fb20d" exitCode=0
May 25 07:03:59.723276 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:59.722787    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" event={"ID":"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b","Type":"ContainerDied","Data":"b3ad88a3d8569ef43ba663607d165b04affa8ccb57e14311c371f2a4ed9fb20d"}
May 25 07:03:59.724921 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:59.724901    2572 generic.go:358] "Generic (PLEG): container finished" podID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerID="10b1d1f5485ed0cb392530857264c6c34f5bd378f64d32bdaa1e162bf1234479" exitCode=0
May 25 07:03:59.724992 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:59.724973    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" event={"ID":"aa90d273-1d8b-4839-9c9b-1fe7241526bb","Type":"ContainerDied","Data":"10b1d1f5485ed0cb392530857264c6c34f5bd378f64d32bdaa1e162bf1234479"}
May 25 07:03:59.725992 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:59.725973    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp" event={"ID":"b4286131-2303-4bed-8c12-7945370edbfa","Type":"ContainerDied","Data":"eff5f1ab02126506d3555d65cf140efdf91dffd00f89a6a37fe3751ea50c6bf8"}
May 25 07:03:59.726083 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:59.726004    2572 scope.go:117] "RemoveContainer" containerID="9a3a01db961e50b1b443da2013fd98e6cfa79d77ba35732f0edec14e7a5d07f0"
May 25 07:03:59.726083 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:59.726016    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"
May 25 07:03:59.744705 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:59.744679    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"]
May 25 07:03:59.752002 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:03:59.749392    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5bdc49556f-lcbbp"]
May 25 07:04:00.281856 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:00.281814    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="b4286131-2303-4bed-8c12-7945370edbfa" path="/var/lib/kubelet/pods/b4286131-2303-4bed-8c12-7945370edbfa/volumes"
May 25 07:04:00.798236 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:00.798196    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"]
May 25 07:04:00.798642 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:00.798388    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr" podUID="6eb5de76-17d3-49e8-940d-5a96e8c6e849" containerName="postgres" containerID="cri-o://c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7" gracePeriod=30
May 25 07:04:00.802592 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:00.802568    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-wpcpw"]
May 25 07:04:00.802749 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:00.802731    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/redis-854544c579-wpcpw" podUID="4d617185-b2c6-45bd-b1f8-bd6578d767bf" containerName="redis" containerID="cri-o://79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318" gracePeriod=30
May 25 07:04:01.049683 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.049617    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"
May 25 07:04:01.073179 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.073156    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-wpcpw"
May 25 07:04:01.150583 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.150539    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pvv68\" (UniqueName: \"kubernetes.io/projected/6eb5de76-17d3-49e8-940d-5a96e8c6e849-kube-api-access-pvv68\") pod \"6eb5de76-17d3-49e8-940d-5a96e8c6e849\" (UID: \"6eb5de76-17d3-49e8-940d-5a96e8c6e849\") "
May 25 07:04:01.150779 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.150629    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/6eb5de76-17d3-49e8-940d-5a96e8c6e849-postgresdata\") pod \"6eb5de76-17d3-49e8-940d-5a96e8c6e849\" (UID: \"6eb5de76-17d3-49e8-940d-5a96e8c6e849\") "
May 25 07:04:01.152870 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.152824    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6eb5de76-17d3-49e8-940d-5a96e8c6e849-kube-api-access-pvv68" (OuterVolumeSpecName: "kube-api-access-pvv68") pod "6eb5de76-17d3-49e8-940d-5a96e8c6e849" (UID: "6eb5de76-17d3-49e8-940d-5a96e8c6e849"). InnerVolumeSpecName "kube-api-access-pvv68". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 07:04:01.179851 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.179814    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6eb5de76-17d3-49e8-940d-5a96e8c6e849-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "6eb5de76-17d3-49e8-940d-5a96e8c6e849" (UID: "6eb5de76-17d3-49e8-940d-5a96e8c6e849"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 07:04:01.251954 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.251916    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w4gtk\" (UniqueName: \"kubernetes.io/projected/4d617185-b2c6-45bd-b1f8-bd6578d767bf-kube-api-access-w4gtk\") pod \"4d617185-b2c6-45bd-b1f8-bd6578d767bf\" (UID: \"4d617185-b2c6-45bd-b1f8-bd6578d767bf\") "
May 25 07:04:01.252127 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.252114    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pvv68\" (UniqueName: \"kubernetes.io/projected/6eb5de76-17d3-49e8-940d-5a96e8c6e849-kube-api-access-pvv68\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:04:01.252172 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.252130    2572 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/6eb5de76-17d3-49e8-940d-5a96e8c6e849-postgresdata\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:04:01.253957 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.253922    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4d617185-b2c6-45bd-b1f8-bd6578d767bf-kube-api-access-w4gtk" (OuterVolumeSpecName: "kube-api-access-w4gtk") pod "4d617185-b2c6-45bd-b1f8-bd6578d767bf" (UID: "4d617185-b2c6-45bd-b1f8-bd6578d767bf"). InnerVolumeSpecName "kube-api-access-w4gtk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 07:04:01.353122 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.353088    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w4gtk\" (UniqueName: \"kubernetes.io/projected/4d617185-b2c6-45bd-b1f8-bd6578d767bf-kube-api-access-w4gtk\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:04:01.733866 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.733767    2572 generic.go:358] "Generic (PLEG): container finished" podID="4d617185-b2c6-45bd-b1f8-bd6578d767bf" containerID="79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318" exitCode=0
May 25 07:04:01.733866 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.733815    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-wpcpw"
May 25 07:04:01.734103 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.733861    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-wpcpw" event={"ID":"4d617185-b2c6-45bd-b1f8-bd6578d767bf","Type":"ContainerDied","Data":"79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318"}
May 25 07:04:01.734103 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.733902    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-wpcpw" event={"ID":"4d617185-b2c6-45bd-b1f8-bd6578d767bf","Type":"ContainerDied","Data":"4ac7f2587604d0f8f0afea030cfdf925795eae462237661b3ad7c321fad03096"}
May 25 07:04:01.734103 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.733921    2572 scope.go:117] "RemoveContainer" containerID="79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318"
May 25 07:04:01.735058 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.735037    2572 generic.go:358] "Generic (PLEG): container finished" podID="6eb5de76-17d3-49e8-940d-5a96e8c6e849" containerID="c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7" exitCode=0
May 25 07:04:01.735196 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.735085    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"
May 25 07:04:01.735196 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.735097    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr" event={"ID":"6eb5de76-17d3-49e8-940d-5a96e8c6e849","Type":"ContainerDied","Data":"c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7"}
May 25 07:04:01.735196 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.735122    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-vnqxr" event={"ID":"6eb5de76-17d3-49e8-940d-5a96e8c6e849","Type":"ContainerDied","Data":"62cbeab672084c8ce0fa341208551dc5e7cd5bf88d0da618c3e883020853c5b5"}
May 25 07:04:01.742806 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.742790    2572 scope.go:117] "RemoveContainer" containerID="79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318"
May 25 07:04:01.743107 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:04:01.743085    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318\": container with ID starting with 79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318 not found: ID does not exist" containerID="79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318"
May 25 07:04:01.743221 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.743112    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318"} err="failed to get container status \"79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318\": rpc error: code = NotFound desc = could not find container \"79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318\": container with ID starting with 79d20d4a2538a95ca77e95c026df462a7074cb8f0cf6098d2d2974d84e7fa318 not found: ID does not exist"
May 25 07:04:01.743221 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.743128    2572 scope.go:117] "RemoveContainer" containerID="c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7"
May 25 07:04:01.750188 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.750170    2572 scope.go:117] "RemoveContainer" containerID="c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7"
May 25 07:04:01.750400 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:04:01.750380    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7\": container with ID starting with c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7 not found: ID does not exist" containerID="c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7"
May 25 07:04:01.750445 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.750408    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7"} err="failed to get container status \"c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7\": rpc error: code = NotFound desc = could not find container \"c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7\": container with ID starting with c643b903d6a0085cd00de56c543bcfdfe4b8080bcc376ff34ca6391be88516a7 not found: ID does not exist"
May 25 07:04:01.752786 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.752763    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-wpcpw"]
May 25 07:04:01.755612 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.755593    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-854544c579-wpcpw"]
May 25 07:04:01.762856 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.762821    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"]
May 25 07:04:01.764960 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:01.764942    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-vnqxr"]
May 25 07:04:02.281530 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:02.281498    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4d617185-b2c6-45bd-b1f8-bd6578d767bf" path="/var/lib/kubelet/pods/4d617185-b2c6-45bd-b1f8-bd6578d767bf/volumes"
May 25 07:04:02.281887 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:02.281825    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6eb5de76-17d3-49e8-940d-5a96e8c6e849" path="/var/lib/kubelet/pods/6eb5de76-17d3-49e8-940d-5a96e8c6e849/volumes"
May 25 07:04:05.038980 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:05.038932    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 25 07:04:06.236177 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:06.236129    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 25 07:04:15.038692 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:15.038648    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 25 07:04:16.235439 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:16.235394    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 25 07:04:25.039028 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:25.038982    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 25 07:04:25.039390 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:25.039116    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:04:26.236169 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:26.236124    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 25 07:04:26.236561 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:26.236238    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:04:28.816854 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:28.816811    2572 generic.go:358] "Generic (PLEG): container finished" podID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerID="3c1f15894aa689562ff023aee002fa9cfa652e8c78a0464df7c61c2f57fb6467" exitCode=137
May 25 07:04:28.817255 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:28.816884    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" event={"ID":"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b","Type":"ContainerDied","Data":"3c1f15894aa689562ff023aee002fa9cfa652e8c78a0464df7c61c2f57fb6467"}
May 25 07:04:28.818898 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:28.818876    2572 generic.go:358] "Generic (PLEG): container finished" podID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerID="a849b4e99a2c61a68654e351f3b0c23a37692d61079af4f1da5cdecc1c0e8945" exitCode=137
May 25 07:04:28.818999 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:28.818945    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" event={"ID":"aa90d273-1d8b-4839-9c9b-1fe7241526bb","Type":"ContainerDied","Data":"a849b4e99a2c61a68654e351f3b0c23a37692d61079af4f1da5cdecc1c0e8945"}
May 25 07:04:29.322127 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.322107    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:04:29.365889 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.365869    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:04:29.457684 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.457593    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/aa90d273-1d8b-4839-9c9b-1fe7241526bb-feast-data\") pod \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") "
May 25 07:04:29.457684 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.457630    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-feast-data\") pod \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") "
May 25 07:04:29.457684 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.457661    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-registry-tls\") pod \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") "
May 25 07:04:29.457981 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.457699    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/aa90d273-1d8b-4839-9c9b-1fe7241526bb-registry-tls\") pod \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") "
May 25 07:04:29.457981 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.457724    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-online-tls\") pod \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") "
May 25 07:04:29.457981 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.457757    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-glvv2\" (UniqueName: \"kubernetes.io/projected/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-kube-api-access-glvv2\") pod \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\" (UID: \"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b\") "
May 25 07:04:29.457981 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.457870    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-xh89w\" (UniqueName: \"kubernetes.io/projected/aa90d273-1d8b-4839-9c9b-1fe7241526bb-kube-api-access-xh89w\") pod \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") "
May 25 07:04:29.457981 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.457925    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/aa90d273-1d8b-4839-9c9b-1fe7241526bb-online-tls\") pod \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\" (UID: \"aa90d273-1d8b-4839-9c9b-1fe7241526bb\") "
May 25 07:04:29.458258 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.458194    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-feast-data" (OuterVolumeSpecName: "feast-data") pod "d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" (UID: "d5022b9a-d40e-48e5-a0e5-a5f088d0a81b"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 07:04:29.460303 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.460273    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" (UID: "d5022b9a-d40e-48e5-a0e5-a5f088d0a81b"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:04:29.460443 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.460323    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-online-tls" (OuterVolumeSpecName: "online-tls") pod "d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" (UID: "d5022b9a-d40e-48e5-a0e5-a5f088d0a81b"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:04:29.460443 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.460417    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-kube-api-access-glvv2" (OuterVolumeSpecName: "kube-api-access-glvv2") pod "d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" (UID: "d5022b9a-d40e-48e5-a0e5-a5f088d0a81b"). InnerVolumeSpecName "kube-api-access-glvv2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 07:04:29.460443 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.460424    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa90d273-1d8b-4839-9c9b-1fe7241526bb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "aa90d273-1d8b-4839-9c9b-1fe7241526bb" (UID: "aa90d273-1d8b-4839-9c9b-1fe7241526bb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:04:29.460592 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.460533    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/aa90d273-1d8b-4839-9c9b-1fe7241526bb-online-tls" (OuterVolumeSpecName: "online-tls") pod "aa90d273-1d8b-4839-9c9b-1fe7241526bb" (UID: "aa90d273-1d8b-4839-9c9b-1fe7241526bb"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:04:29.460644 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.460601    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa90d273-1d8b-4839-9c9b-1fe7241526bb-kube-api-access-xh89w" (OuterVolumeSpecName: "kube-api-access-xh89w") pod "aa90d273-1d8b-4839-9c9b-1fe7241526bb" (UID: "aa90d273-1d8b-4839-9c9b-1fe7241526bb"). InnerVolumeSpecName "kube-api-access-xh89w". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 07:04:29.468206 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.468180    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa90d273-1d8b-4839-9c9b-1fe7241526bb-feast-data" (OuterVolumeSpecName: "feast-data") pod "aa90d273-1d8b-4839-9c9b-1fe7241526bb" (UID: "aa90d273-1d8b-4839-9c9b-1fe7241526bb"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 07:04:29.558660 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.558631    2572 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/aa90d273-1d8b-4839-9c9b-1fe7241526bb-feast-data\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:04:29.558660 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.558660    2572 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-feast-data\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:04:29.558816 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.558672    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:04:29.558816 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.558681    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/aa90d273-1d8b-4839-9c9b-1fe7241526bb-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:04:29.558816 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.558691    2572 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-online-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:04:29.558816 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.558700    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-glvv2\" (UniqueName: \"kubernetes.io/projected/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b-kube-api-access-glvv2\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:04:29.558816 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.558709    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-xh89w\" (UniqueName: \"kubernetes.io/projected/aa90d273-1d8b-4839-9c9b-1fe7241526bb-kube-api-access-xh89w\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:04:29.558816 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.558718    2572 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/aa90d273-1d8b-4839-9c9b-1fe7241526bb-online-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:04:29.824271 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.824237    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg" event={"ID":"d5022b9a-d40e-48e5-a0e5-a5f088d0a81b","Type":"ContainerDied","Data":"3f5602092409f7025af849f08065301ba2c841a91806bd8c8ac65a1317812e0a"}
May 25 07:04:29.824718 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.824284    2572 scope.go:117] "RemoveContainer" containerID="b3ad88a3d8569ef43ba663607d165b04affa8ccb57e14311c371f2a4ed9fb20d"
May 25 07:04:29.824718 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.824256    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"
May 25 07:04:29.831836 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.831809    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw" event={"ID":"aa90d273-1d8b-4839-9c9b-1fe7241526bb","Type":"ContainerDied","Data":"aba1dcc9fe1e7cd64cb89c8c8d273ba69f6a8e16dcc2903a99acf5b1844b39a9"}
May 25 07:04:29.831968 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.831905    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"
May 25 07:04:29.838573 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.838551    2572 scope.go:117] "RemoveContainer" containerID="3c1f15894aa689562ff023aee002fa9cfa652e8c78a0464df7c61c2f57fb6467"
May 25 07:04:29.845671 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.845653    2572 scope.go:117] "RemoveContainer" containerID="fb703e24d51b88749db5727eb8c2ff5e8516cf91e973375aa354e723186ede54"
May 25 07:04:29.847939 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.847909    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"]
May 25 07:04:29.851338 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.851318    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-54f4667468-wwpvg"]
May 25 07:04:29.854045 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.854031    2572 scope.go:117] "RemoveContainer" containerID="abeeb8344f056a779cf8c6dc300824541b169de85d34564221dd6f41ccb47445"
May 25 07:04:29.860699 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.860672    2572 scope.go:117] "RemoveContainer" containerID="10b1d1f5485ed0cb392530857264c6c34f5bd378f64d32bdaa1e162bf1234479"
May 25 07:04:29.860774 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.860753    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"]
May 25 07:04:29.863583 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.863565    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6bcfb77767-6hbzw"]
May 25 07:04:29.867659 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.867634    2572 scope.go:117] "RemoveContainer" containerID="a849b4e99a2c61a68654e351f3b0c23a37692d61079af4f1da5cdecc1c0e8945"
May 25 07:04:29.875035 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.875018    2572 scope.go:117] "RemoveContainer" containerID="211c66cf6d99f0b4b9c53fd957ee72108923098ea2d4dc23f4dbe5858362e7b3"
May 25 07:04:29.881387 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:29.881369    2572 scope.go:117] "RemoveContainer" containerID="b960325924d20cc190cbf5e42acbb796a694b89affb93d472c974865da86e2c2"
May 25 07:04:30.282643 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:30.282554    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" path="/var/lib/kubelet/pods/aa90d273-1d8b-4839-9c9b-1fe7241526bb/volumes"
May 25 07:04:30.283182 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:30.283159    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" path="/var/lib/kubelet/pods/d5022b9a-d40e-48e5-a0e5-a5f088d0a81b/volumes"
May 25 07:04:46.964294 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964256    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"]
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964532    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="registry"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964543    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="registry"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964553    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="b4286131-2303-4bed-8c12-7945370edbfa" containerName="manager"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964559    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="b4286131-2303-4bed-8c12-7945370edbfa" containerName="manager"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964567    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="feast-apply"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964572    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="feast-apply"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964577    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="online"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964584    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="online"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964590    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="registry"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964595    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="registry"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964601    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="online"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964606    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="online"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964612    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="feast-init"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964617    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="feast-init"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964626    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4d617185-b2c6-45bd-b1f8-bd6578d767bf" containerName="redis"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964631    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="4d617185-b2c6-45bd-b1f8-bd6578d767bf" containerName="redis"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964636    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6eb5de76-17d3-49e8-940d-5a96e8c6e849" containerName="postgres"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964641    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="6eb5de76-17d3-49e8-940d-5a96e8c6e849" containerName="postgres"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964646    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="feast-init"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964651    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="feast-init"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964657    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="feast-apply"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964662    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="feast-apply"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964707    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="online"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964715    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="4d617185-b2c6-45bd-b1f8-bd6578d767bf" containerName="redis"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964721    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="registry"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964728    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa90d273-1d8b-4839-9c9b-1fe7241526bb" containerName="registry"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964736    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="d5022b9a-d40e-48e5-a0e5-a5f088d0a81b" containerName="online"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964743    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="6eb5de76-17d3-49e8-940d-5a96e8c6e849" containerName="postgres"
May 25 07:04:46.964745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.964750    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="b4286131-2303-4bed-8c12-7945370edbfa" containerName="manager"
May 25 07:04:46.970215 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.970194    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:46.971998 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.971977    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 25 07:04:46.972300 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.972269    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 25 07:04:46.972300 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.972286    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 25 07:04:46.972459 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.972419    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 25 07:04:46.972459 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.972435    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 25 07:04:46.972734 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.972719    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 25 07:04:46.972782 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.972744    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-b9wsg\""
May 25 07:04:46.976630 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.976604    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"]
May 25 07:04:46.986442 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.986422    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab8689e1-a326-456a-9298-2a17174f9f43-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:46.986549 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.986448    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:46.986549 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.986473    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:46.986549 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.986546    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wvlf8\" (UniqueName: \"kubernetes.io/projected/ab8689e1-a326-456a-9298-2a17174f9f43-kube-api-access-wvlf8\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:46.986665 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.986578    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:46.986665 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:46.986606    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.088016 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.087983    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab8689e1-a326-456a-9298-2a17174f9f43-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.088016 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.088020    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.088247 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.088045    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.088247 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.088071    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wvlf8\" (UniqueName: \"kubernetes.io/projected/ab8689e1-a326-456a-9298-2a17174f9f43-kube-api-access-wvlf8\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.088247 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.088097    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.088247 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.088134    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.088470 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.088448    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab8689e1-a326-456a-9298-2a17174f9f43-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.090590 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.090567    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.090725 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.090612    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.090725 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.090673    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.090725 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.090699    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.094476 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.094458    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wvlf8\" (UniqueName: \"kubernetes.io/projected/ab8689e1-a326-456a-9298-2a17174f9f43-kube-api-access-wvlf8\") pod \"feast-simple-feast-setup-664fcfcc58-zs222\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.291569 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.291484    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:04:47.417509 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.417479    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"]
May 25 07:04:47.420518 ip-10-0-128-6 kubenswrapper[2572]: W0525 07:04:47.420492    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab8689e1_a326_456a_9298_2a17174f9f43.slice/crio-88ffa80d39b32f53757ff0fd8a2e6426528273415d1f70b4c86da3a6114ba06b WatchSource:0}: Error finding container 88ffa80d39b32f53757ff0fd8a2e6426528273415d1f70b4c86da3a6114ba06b: Status 404 returned error can't find the container with id 88ffa80d39b32f53757ff0fd8a2e6426528273415d1f70b4c86da3a6114ba06b
May 25 07:04:47.422180 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.422166    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 07:04:47.882202 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:04:47.882170    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerStarted","Data":"88ffa80d39b32f53757ff0fd8a2e6426528273415d1f70b4c86da3a6114ba06b"}
May 25 07:05:01.929773 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:01.929726    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerStarted","Data":"c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784"}
May 25 07:05:06.946021 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:06.945928    2572 generic.go:358] "Generic (PLEG): container finished" podID="ab8689e1-a326-456a-9298-2a17174f9f43" containerID="c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784" exitCode=0
May 25 07:05:06.946021 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:06.946006    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerDied","Data":"c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784"}
May 25 07:05:07.951008 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:07.950964    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerStarted","Data":"9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d"}
May 25 07:05:12.967003 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:12.966969    2572 generic.go:358] "Generic (PLEG): container finished" podID="ab8689e1-a326-456a-9298-2a17174f9f43" containerID="9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d" exitCode=0
May 25 07:05:12.967479 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:12.967053    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerDied","Data":"9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d"}
May 25 07:05:13.980978 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:13.980936    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerStarted","Data":"20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac"}
May 25 07:05:13.981510 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:13.980992    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerStarted","Data":"4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3"}
May 25 07:05:13.981510 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:13.981008    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerStarted","Data":"f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56"}
May 25 07:05:13.981510 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:13.981025    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerStarted","Data":"71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0"}
May 25 07:05:14.004959 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:14.004872    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podStartSLOduration=13.902867456 podStartE2EDuration="28.004820479s" podCreationTimestamp="2026-05-25 07:04:46 +0000 UTC" firstStartedPulling="2026-05-25 07:04:47.422294948 +0000 UTC m=+1125.658481318" lastFinishedPulling="2026-05-25 07:05:01.524247968 +0000 UTC m=+1139.760434341" observedRunningTime="2026-05-25 07:05:14.001021794 +0000 UTC m=+1152.237208209" watchObservedRunningTime="2026-05-25 07:05:14.004820479 +0000 UTC m=+1152.241006872"
May 25 07:05:14.292593 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:14.292487    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:14.292593 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:14.292535    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:14.292866 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:14.292657    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:14.292866 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:14.292691    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:14.295125 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:14.294967    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.34:8816: connect: connection refused"
May 25 07:05:14.295125 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:14.295003    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.34:6571: connect: connection refused"
May 25 07:05:14.295125 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:14.294999    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 25 07:05:14.295125 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:14.295066    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 25 07:05:17.292281 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:17.292237    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:17.292798 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:17.292768    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.34:6571: connect: connection refused"
May 25 07:05:17.292962 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:17.292816    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:17.292962 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:17.292870    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:17.292962 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:17.292886    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:17.292962 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:17.292709    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 25 07:05:17.292962 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:17.292565    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 25 07:05:17.293677 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:17.292568    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.34:8816: connect: connection refused"
May 25 07:05:20.293129 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:20.293101    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:20.293672 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:20.293197    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:20.293672 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:20.293498    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:20.294008 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:20.293975    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:20.294124 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:20.294057    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:20.294407 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:20.294389    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:20.297579 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:20.297562    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:05:21.010577 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:05:21.010547    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:06:02.281227 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:02.281148    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 07:06:02.284901 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:02.284880    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 07:06:23.353198 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.353162    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"]
May 25 07:06:23.353673 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.353610    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="registry" containerID="cri-o://71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0" gracePeriod=30
May 25 07:06:23.353744 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.353662    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="offline" containerID="cri-o://4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3" gracePeriod=30
May 25 07:06:23.353744 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.353689    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="ui" containerID="cri-o://20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac" gracePeriod=30
May 25 07:06:23.353869 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.353749    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="online" containerID="cri-o://f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56" gracePeriod=30
May 25 07:06:23.610624 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.610539    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"]
May 25 07:06:23.614408 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.614389    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.616169 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.616133    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.616293 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.616211    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.616293 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.616254    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-gswbv\""
May 25 07:06:23.616293 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.616264    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.616412 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.616295    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.616412 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.616334    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.616412 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.616358    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-llvzd\" (UniqueName: \"kubernetes.io/projected/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-kube-api-access-llvzd\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.624153 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.624125    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"]
May 25 07:06:23.716994 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.716954    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.717202 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.717013    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.717202 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.717035    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-llvzd\" (UniqueName: \"kubernetes.io/projected/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-kube-api-access-llvzd\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.717202 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:06:23.717093    2572 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 25 07:06:23.717202 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.717118    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.717202 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:06:23.717160    2572 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 25 07:06:23.717202 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:06:23.717171    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-offline-tls podName:83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2 nodeName:}" failed. No retries permitted until 2026-05-25 07:06:24.217148702 +0000 UTC m=+1222.453335090 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-offline-tls") pod "feast-simple-feast-setup-664fcfcc58-gnprz" (UID: "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2") : secret "feast-simple-feast-setup-offline-tls" not found
May 25 07:06:23.717536 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:06:23.717236    2572 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 25 07:06:23.717536 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:06:23.717239    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-ui-tls podName:83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2 nodeName:}" failed. No retries permitted until 2026-05-25 07:06:24.2172195 +0000 UTC m=+1222.453405880 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-gnprz" (UID: "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2") : secret "feast-simple-feast-setup-ui-tls" not found
May 25 07:06:23.717536 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:06:23.717308    2572 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-registry-tls podName:83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2 nodeName:}" failed. No retries permitted until 2026-05-25 07:06:24.217277907 +0000 UTC m=+1222.453464283 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-gnprz" (UID: "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2") : secret "feast-simple-feast-setup-registry-tls" not found
May 25 07:06:23.717536 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.717356    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.717536 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.717388    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.717798 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.717738    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.720067 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.720029    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:23.724557 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:23.724529    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-llvzd\" (UniqueName: \"kubernetes.io/projected/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-kube-api-access-llvzd\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:24.183212 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:24.183122    2572 generic.go:358] "Generic (PLEG): container finished" podID="ab8689e1-a326-456a-9298-2a17174f9f43" containerID="20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac" exitCode=0
May 25 07:06:24.183212 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:24.183149    2572 generic.go:358] "Generic (PLEG): container finished" podID="ab8689e1-a326-456a-9298-2a17174f9f43" containerID="f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56" exitCode=0
May 25 07:06:24.183212 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:24.183192    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerDied","Data":"20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac"}
May 25 07:06:24.183412 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:24.183223    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerDied","Data":"f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56"}
May 25 07:06:24.220231 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:24.220198    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:24.220341 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:24.220242    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:24.220341 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:24.220300    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:24.222636 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:24.222612    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:24.222737 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:24.222647    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:24.223868 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:24.223253    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-gnprz\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:24.226621 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:24.226051    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:24.358470 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:24.358432    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"]
May 25 07:06:24.361283 ip-10-0-128-6 kubenswrapper[2572]: W0525 07:06:24.361254    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod83dfdf28_aae7_4406_b0e5_3e7a0e85c4f2.slice/crio-e8bf1cffb1874f0111d5ab7a9e5acb7f15c293160d449842ac67062937abc57d WatchSource:0}: Error finding container e8bf1cffb1874f0111d5ab7a9e5acb7f15c293160d449842ac67062937abc57d: Status 404 returned error can't find the container with id e8bf1cffb1874f0111d5ab7a9e5acb7f15c293160d449842ac67062937abc57d
May 25 07:06:25.187739 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:25.187699    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerStarted","Data":"c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8"}
May 25 07:06:25.187739 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:25.187744    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerStarted","Data":"e8bf1cffb1874f0111d5ab7a9e5acb7f15c293160d449842ac67062937abc57d"}
May 25 07:06:29.200077 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:29.200040    2572 generic.go:358] "Generic (PLEG): container finished" podID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerID="c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8" exitCode=0
May 25 07:06:29.200459 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:29.200115    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerDied","Data":"c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8"}
May 25 07:06:30.205580 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:30.205548    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerStarted","Data":"f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea"}
May 25 07:06:30.294560 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:30.294521    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 25 07:06:31.007777 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:31.007727    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 25 07:06:35.222214 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:35.222180    2572 generic.go:358] "Generic (PLEG): container finished" podID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerID="f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea" exitCode=0
May 25 07:06:35.222634 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:35.222237    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerDied","Data":"f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea"}
May 25 07:06:36.230399 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:36.230356    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerStarted","Data":"5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2"}
May 25 07:06:36.230911 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:36.230408    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerStarted","Data":"383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94"}
May 25 07:06:36.230911 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:36.230423    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerStarted","Data":"00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75"}
May 25 07:06:36.230911 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:36.230437    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerStarted","Data":"e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e"}
May 25 07:06:36.255548 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:36.255433    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podStartSLOduration=13.255410053 podStartE2EDuration="13.255410053s" podCreationTimestamp="2026-05-25 07:06:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 07:06:36.251547632 +0000 UTC m=+1234.487734054" watchObservedRunningTime="2026-05-25 07:06:36.255410053 +0000 UTC m=+1234.491596457"
May 25 07:06:39.227147 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:39.227064    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:39.227147 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:39.227117    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:39.227147 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:39.227131    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:39.227147 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:39.227143    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:39.229390 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:39.229219    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 25 07:06:39.229390 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:39.229262    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
May 25 07:06:39.229390 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:39.229294    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 25 07:06:39.229390 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:39.229299    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.35:8816: connect: connection refused"
May 25 07:06:40.294424 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:40.294381    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 25 07:06:41.008538 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:41.008495    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 25 07:06:42.228217 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:42.228188    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:42.228692 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:42.228456    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:42.228801 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:42.228776    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:42.228887 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:42.228804    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:42.228961 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:42.228949    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:42.229001 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:42.228993    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:42.229035 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:42.229023    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:42.232781 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:42.232766    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:42.251908 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:42.251881    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:42.251908 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:42.251905    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:42.252450 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:42.252433    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:42.254674 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:42.254658    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:06:50.294433 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:50.294389    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 25 07:06:50.296835 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:50.294559    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:06:51.007644 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:51.007603    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 25 07:06:51.007827 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:51.007741    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:06:54.005963 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.005936    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:06:54.081810 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.081773    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-offline-tls\") pod \"ab8689e1-a326-456a-9298-2a17174f9f43\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") "
May 25 07:06:54.081810 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.081814    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-online-tls\") pod \"ab8689e1-a326-456a-9298-2a17174f9f43\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") "
May 25 07:06:54.082060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.081878    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-registry-tls\") pod \"ab8689e1-a326-456a-9298-2a17174f9f43\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") "
May 25 07:06:54.082060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.081902    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-ui-tls\") pod \"ab8689e1-a326-456a-9298-2a17174f9f43\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") "
May 25 07:06:54.082060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.081923    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab8689e1-a326-456a-9298-2a17174f9f43-feast-data\") pod \"ab8689e1-a326-456a-9298-2a17174f9f43\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") "
May 25 07:06:54.082060 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.081975    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wvlf8\" (UniqueName: \"kubernetes.io/projected/ab8689e1-a326-456a-9298-2a17174f9f43-kube-api-access-wvlf8\") pod \"ab8689e1-a326-456a-9298-2a17174f9f43\" (UID: \"ab8689e1-a326-456a-9298-2a17174f9f43\") "
May 25 07:06:54.082628 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.082518    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab8689e1-a326-456a-9298-2a17174f9f43-feast-data" (OuterVolumeSpecName: "feast-data") pod "ab8689e1-a326-456a-9298-2a17174f9f43" (UID: "ab8689e1-a326-456a-9298-2a17174f9f43"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 07:06:54.084312 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.084282    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ab8689e1-a326-456a-9298-2a17174f9f43" (UID: "ab8689e1-a326-456a-9298-2a17174f9f43"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:06:54.084429 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.084354    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ab8689e1-a326-456a-9298-2a17174f9f43" (UID: "ab8689e1-a326-456a-9298-2a17174f9f43"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:06:54.084429 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.084371    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab8689e1-a326-456a-9298-2a17174f9f43-kube-api-access-wvlf8" (OuterVolumeSpecName: "kube-api-access-wvlf8") pod "ab8689e1-a326-456a-9298-2a17174f9f43" (UID: "ab8689e1-a326-456a-9298-2a17174f9f43"). InnerVolumeSpecName "kube-api-access-wvlf8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 07:06:54.084536 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.084470    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-online-tls" (OuterVolumeSpecName: "online-tls") pod "ab8689e1-a326-456a-9298-2a17174f9f43" (UID: "ab8689e1-a326-456a-9298-2a17174f9f43"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:06:54.084536 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.084497    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ab8689e1-a326-456a-9298-2a17174f9f43" (UID: "ab8689e1-a326-456a-9298-2a17174f9f43"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:06:54.183199 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.183113    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wvlf8\" (UniqueName: \"kubernetes.io/projected/ab8689e1-a326-456a-9298-2a17174f9f43-kube-api-access-wvlf8\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:06:54.183199 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.183143    2572 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-offline-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:06:54.183199 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.183153    2572 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-online-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:06:54.183199 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.183161    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:06:54.183199 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.183170    2572 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab8689e1-a326-456a-9298-2a17174f9f43-ui-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:06:54.183199 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.183177    2572 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab8689e1-a326-456a-9298-2a17174f9f43-feast-data\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:06:54.292120 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.292091    2572 generic.go:358] "Generic (PLEG): container finished" podID="ab8689e1-a326-456a-9298-2a17174f9f43" containerID="4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3" exitCode=137
May 25 07:06:54.292120 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.292117    2572 generic.go:358] "Generic (PLEG): container finished" podID="ab8689e1-a326-456a-9298-2a17174f9f43" containerID="71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0" exitCode=137
May 25 07:06:54.292275 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.292162    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"
May 25 07:06:54.292275 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.292169    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerDied","Data":"4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3"}
May 25 07:06:54.292275 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.292210    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerDied","Data":"71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0"}
May 25 07:06:54.292275 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.292222    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222" event={"ID":"ab8689e1-a326-456a-9298-2a17174f9f43","Type":"ContainerDied","Data":"88ffa80d39b32f53757ff0fd8a2e6426528273415d1f70b4c86da3a6114ba06b"}
May 25 07:06:54.292275 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.292236    2572 scope.go:117] "RemoveContainer" containerID="20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac"
May 25 07:06:54.300488 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.300470    2572 scope.go:117] "RemoveContainer" containerID="4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3"
May 25 07:06:54.307296 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.307275    2572 scope.go:117] "RemoveContainer" containerID="f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56"
May 25 07:06:54.308706 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.308684    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"]
May 25 07:06:54.314137 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.314113    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zs222"]
May 25 07:06:54.315654 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.315638    2572 scope.go:117] "RemoveContainer" containerID="71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0"
May 25 07:06:54.322527 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.322509    2572 scope.go:117] "RemoveContainer" containerID="9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d"
May 25 07:06:54.329330 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.329311    2572 scope.go:117] "RemoveContainer" containerID="c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784"
May 25 07:06:54.336087 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.336071    2572 scope.go:117] "RemoveContainer" containerID="20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac"
May 25 07:06:54.336328 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:06:54.336309    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac\": container with ID starting with 20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac not found: ID does not exist" containerID="20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac"
May 25 07:06:54.336378 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.336337    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac"} err="failed to get container status \"20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac\": rpc error: code = NotFound desc = could not find container \"20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac\": container with ID starting with 20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac not found: ID does not exist"
May 25 07:06:54.336378 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.336365    2572 scope.go:117] "RemoveContainer" containerID="4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3"
May 25 07:06:54.336590 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:06:54.336572    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3\": container with ID starting with 4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3 not found: ID does not exist" containerID="4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3"
May 25 07:06:54.336632 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.336596    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3"} err="failed to get container status \"4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3\": rpc error: code = NotFound desc = could not find container \"4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3\": container with ID starting with 4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3 not found: ID does not exist"
May 25 07:06:54.336632 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.336614    2572 scope.go:117] "RemoveContainer" containerID="f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56"
May 25 07:06:54.336873 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:06:54.336853    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56\": container with ID starting with f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56 not found: ID does not exist" containerID="f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56"
May 25 07:06:54.336982 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.336877    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56"} err="failed to get container status \"f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56\": rpc error: code = NotFound desc = could not find container \"f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56\": container with ID starting with f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56 not found: ID does not exist"
May 25 07:06:54.336982 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.336892    2572 scope.go:117] "RemoveContainer" containerID="71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0"
May 25 07:06:54.337123 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:06:54.337109    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0\": container with ID starting with 71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0 not found: ID does not exist" containerID="71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0"
May 25 07:06:54.337165 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.337125    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0"} err="failed to get container status \"71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0\": rpc error: code = NotFound desc = could not find container \"71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0\": container with ID starting with 71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0 not found: ID does not exist"
May 25 07:06:54.337165 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.337137    2572 scope.go:117] "RemoveContainer" containerID="9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d"
May 25 07:06:54.337354 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:06:54.337335    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d\": container with ID starting with 9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d not found: ID does not exist" containerID="9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d"
May 25 07:06:54.337393 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.337361    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d"} err="failed to get container status \"9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d\": rpc error: code = NotFound desc = could not find container \"9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d\": container with ID starting with 9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d not found: ID does not exist"
May 25 07:06:54.337393 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.337377    2572 scope.go:117] "RemoveContainer" containerID="c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784"
May 25 07:06:54.337599 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:06:54.337579    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784\": container with ID starting with c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784 not found: ID does not exist" containerID="c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784"
May 25 07:06:54.337645 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.337602    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784"} err="failed to get container status \"c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784\": rpc error: code = NotFound desc = could not find container \"c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784\": container with ID starting with c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784 not found: ID does not exist"
May 25 07:06:54.337645 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.337614    2572 scope.go:117] "RemoveContainer" containerID="20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac"
May 25 07:06:54.337802 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.337783    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac"} err="failed to get container status \"20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac\": rpc error: code = NotFound desc = could not find container \"20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac\": container with ID starting with 20128a42f67ed623562bcf45d1cdcd1d71cae871bda0738b70e2881bd5b7ffac not found: ID does not exist"
May 25 07:06:54.337858 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.337803    2572 scope.go:117] "RemoveContainer" containerID="4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3"
May 25 07:06:54.338022 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.338003    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3"} err="failed to get container status \"4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3\": rpc error: code = NotFound desc = could not find container \"4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3\": container with ID starting with 4a80ca856c01ffcd9e457fd960a8435aed9b3df783a7dc8363f953fa03491bd3 not found: ID does not exist"
May 25 07:06:54.338064 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.338022    2572 scope.go:117] "RemoveContainer" containerID="f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56"
May 25 07:06:54.338206 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.338184    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56"} err="failed to get container status \"f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56\": rpc error: code = NotFound desc = could not find container \"f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56\": container with ID starting with f0dd2bf1779016cceae4d9a4c7e5d45278d4df63170dd6936d2f3ca2a934db56 not found: ID does not exist"
May 25 07:06:54.338246 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.338207    2572 scope.go:117] "RemoveContainer" containerID="71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0"
May 25 07:06:54.338377 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.338356    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0"} err="failed to get container status \"71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0\": rpc error: code = NotFound desc = could not find container \"71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0\": container with ID starting with 71a5d1e066e064dbdf341e09d010fab9c5ef0f3d89ecc00d1c4ebd337fc151b0 not found: ID does not exist"
May 25 07:06:54.338422 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.338377    2572 scope.go:117] "RemoveContainer" containerID="9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d"
May 25 07:06:54.338546 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.338526    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d"} err="failed to get container status \"9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d\": rpc error: code = NotFound desc = could not find container \"9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d\": container with ID starting with 9bbd3a1c50d13548bf8aac13b0fe392a7b7ee3f01cea584fe9b7db3d501c136d not found: ID does not exist"
May 25 07:06:54.338596 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.338546    2572 scope.go:117] "RemoveContainer" containerID="c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784"
May 25 07:06:54.338752 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:54.338735    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784"} err="failed to get container status \"c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784\": rpc error: code = NotFound desc = could not find container \"c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784\": container with ID starting with c04beb17af93fb1657136051a7c794f1261048c030c09284677c11253432b784 not found: ID does not exist"
May 25 07:06:56.282452 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:06:56.282422    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" path="/var/lib/kubelet/pods/ab8689e1-a326-456a-9298-2a17174f9f43/volumes"
May 25 07:07:45.447805 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.447765    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"]
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448071    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="feast-init"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448081    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="feast-init"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448095    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="online"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448100    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="online"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448106    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="offline"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448112    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="offline"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448118    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="registry"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448123    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="registry"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448139    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="feast-apply"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448145    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="feast-apply"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448151    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="ui"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448156    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="ui"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448211    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="offline"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448220    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="registry"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448227    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="ui"
May 25 07:07:45.448301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.448234    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="ab8689e1-a326-456a-9298-2a17174f9f43" containerName="online"
May 25 07:07:45.451526 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.451509    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.453496 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.453474    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 25 07:07:45.453626 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.453525    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-9gnmt\""
May 25 07:07:45.453626 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.453587    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 25 07:07:45.453831 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.453792    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 25 07:07:45.453831 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.453807    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 25 07:07:45.454014 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.453833    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 25 07:07:45.454014 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.453859    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 25 07:07:45.460103 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.460082    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"]
May 25 07:07:45.601498 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.601455    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cp24h\" (UniqueName: \"kubernetes.io/projected/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-kube-api-access-cp24h\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.601686 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.601506    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.601686 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.601588    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.601686 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.601620    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.601686 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.601672    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.601881 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.601696    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.702583 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.702492    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.702583 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.702542    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.702803 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.702602    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cp24h\" (UniqueName: \"kubernetes.io/projected/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-kube-api-access-cp24h\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.702803 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.702631    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.702803 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.702660    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.702803 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.702677    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.703124 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.703100    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.703348 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.703321    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.705212 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.705184    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.705297 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.705231    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.705297 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.705231    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.709400 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.709372    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cp24h\" (UniqueName: \"kubernetes.io/projected/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-kube-api-access-cp24h\") pod \"feast-simple-feast-remote-setup-5cff94746b-lmgqb\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.761497 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.761463    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:07:45.893977 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:45.893943    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"]
May 25 07:07:45.897132 ip-10-0-128-6 kubenswrapper[2572]: W0525 07:07:45.897105    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poddebbb6fc_d5d5_4d80_9ea9_94a7332f17c4.slice/crio-e394f9484a9d2d0260bf581010242129adfaed4ac4a942dc40fd9f1d74d2806c WatchSource:0}: Error finding container e394f9484a9d2d0260bf581010242129adfaed4ac4a942dc40fd9f1d74d2806c: Status 404 returned error can't find the container with id e394f9484a9d2d0260bf581010242129adfaed4ac4a942dc40fd9f1d74d2806c
May 25 07:07:46.442120 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:46.442084    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" event={"ID":"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4","Type":"ContainerStarted","Data":"10d8ebabfa184eea72bf20d8f8a2572c4f5307c52f5121901197aaf1993f47dc"}
May 25 07:07:46.442120 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:46.442123    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" event={"ID":"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4","Type":"ContainerStarted","Data":"e394f9484a9d2d0260bf581010242129adfaed4ac4a942dc40fd9f1d74d2806c"}
May 25 07:07:51.457809 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:51.457773    2572 generic.go:358] "Generic (PLEG): container finished" podID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerID="10d8ebabfa184eea72bf20d8f8a2572c4f5307c52f5121901197aaf1993f47dc" exitCode=0
May 25 07:07:51.458281 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:51.457861    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" event={"ID":"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4","Type":"ContainerDied","Data":"10d8ebabfa184eea72bf20d8f8a2572c4f5307c52f5121901197aaf1993f47dc"}
May 25 07:07:52.463294 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:52.463257    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" event={"ID":"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4","Type":"ContainerStarted","Data":"db70b93e094466a5be17d0b2a23983d7fecaf21f0a9e670463a16d127b1d0aff"}
May 25 07:07:57.478454 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:57.478361    2572 generic.go:358] "Generic (PLEG): container finished" podID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerID="db70b93e094466a5be17d0b2a23983d7fecaf21f0a9e670463a16d127b1d0aff" exitCode=0
May 25 07:07:57.478454 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:57.478409    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" event={"ID":"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4","Type":"ContainerDied","Data":"db70b93e094466a5be17d0b2a23983d7fecaf21f0a9e670463a16d127b1d0aff"}
May 25 07:07:58.485029 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:58.484989    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" event={"ID":"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4","Type":"ContainerStarted","Data":"8d9db5e3b2ac35648c13882fc752ff73b020c310318242f7813442c8ca19eacd"}
May 25 07:07:58.485401 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:58.485046    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" event={"ID":"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4","Type":"ContainerStarted","Data":"dc26945181d0268adebf19a43e2a14bec6174a77432c393adc7ddbf18ecd4fdb"}
May 25 07:07:58.485401 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:58.485063    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" event={"ID":"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4","Type":"ContainerStarted","Data":"15bb892988e41f4bfdeb702e7d1b9e2601e3e57cdc946005f2c9d2b0fe6ec3f1"}
May 25 07:07:58.505792 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:07:58.505738    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podStartSLOduration=13.50571844 podStartE2EDuration="13.50571844s" podCreationTimestamp="2026-05-25 07:07:45 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 07:07:58.503111371 +0000 UTC m=+1316.739297763" watchObservedRunningTime="2026-05-25 07:07:58.50571844 +0000 UTC m=+1316.741904833"
May 25 07:08:00.762726 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:00.762683    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:08:00.763195 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:00.762738    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:08:00.763195 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:00.762753    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:08:00.764740 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:00.764712    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.36:8816: connect: connection refused"
May 25 07:08:00.764857 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:00.764748    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 25 07:08:00.764857 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:00.764706    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 25 07:08:03.763407 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:03.763378    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:08:03.763810 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:03.763653    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:08:03.763892 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:03.763872    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:08:03.764173 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:03.764155    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:08:03.768098 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:03.768075    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:08:04.504230 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:04.504197    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:08:04.504230 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:04.504235    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:08:04.507213 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:04.507189    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:08:05.507438 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:08:05.507411    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:09:06.708075 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:06.708036    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"]
May 25 07:09:06.708569 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:06.708426    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="online" containerID="cri-o://15bb892988e41f4bfdeb702e7d1b9e2601e3e57cdc946005f2c9d2b0fe6ec3f1" gracePeriod=30
May 25 07:09:06.708645 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:06.708573    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="offline" containerID="cri-o://dc26945181d0268adebf19a43e2a14bec6174a77432c393adc7ddbf18ecd4fdb" gracePeriod=30
May 25 07:09:06.708645 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:06.708576    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="ui" containerID="cri-o://8d9db5e3b2ac35648c13882fc752ff73b020c310318242f7813442c8ca19eacd" gracePeriod=30
May 25 07:09:06.855776 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:06.855737    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"]
May 25 07:09:06.856140 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:06.856088    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="registry" containerID="cri-o://e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e" gracePeriod=30
May 25 07:09:06.856356 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:06.856149    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="offline" containerID="cri-o://383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94" gracePeriod=30
May 25 07:09:06.856458 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:06.856179    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="ui" containerID="cri-o://5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2" gracePeriod=30
May 25 07:09:06.856458 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:06.856177    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="online" containerID="cri-o://00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75" gracePeriod=30
May 25 07:09:07.685165 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:07.685134    2572 generic.go:358] "Generic (PLEG): container finished" podID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerID="5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2" exitCode=0
May 25 07:09:07.685165 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:07.685160    2572 generic.go:358] "Generic (PLEG): container finished" podID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerID="00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75" exitCode=0
May 25 07:09:07.685391 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:07.685216    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerDied","Data":"5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2"}
May 25 07:09:07.685391 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:07.685261    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerDied","Data":"00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75"}
May 25 07:09:07.687452 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:07.687430    2572 generic.go:358] "Generic (PLEG): container finished" podID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerID="8d9db5e3b2ac35648c13882fc752ff73b020c310318242f7813442c8ca19eacd" exitCode=0
May 25 07:09:07.687452 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:07.687451    2572 generic.go:358] "Generic (PLEG): container finished" podID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerID="15bb892988e41f4bfdeb702e7d1b9e2601e3e57cdc946005f2c9d2b0fe6ec3f1" exitCode=0
May 25 07:09:07.687606 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:07.687503    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" event={"ID":"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4","Type":"ContainerDied","Data":"8d9db5e3b2ac35648c13882fc752ff73b020c310318242f7813442c8ca19eacd"}
May 25 07:09:07.687606 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:07.687539    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" event={"ID":"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4","Type":"ContainerDied","Data":"15bb892988e41f4bfdeb702e7d1b9e2601e3e57cdc946005f2c9d2b0fe6ec3f1"}
May 25 07:09:12.252310 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:12.252269    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 25 07:09:12.252707 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:12.252264    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 25 07:09:13.764359 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:13.764319    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 25 07:09:14.505254 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:14.505209    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 25 07:09:22.252876 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:22.252810    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 25 07:09:22.253352 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:22.252868    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 25 07:09:23.764864 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:23.764810    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 25 07:09:24.505353 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:24.505304    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 25 07:09:32.252994 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:32.252948    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 25 07:09:32.252994 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:32.252969    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 25 07:09:32.253416 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:32.253072    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:09:32.253416 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:32.253113    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:09:33.764213 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:33.764175    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 25 07:09:33.764556 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:33.764316    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:09:34.504660 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:34.504610    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 25 07:09:34.504878 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:34.504737    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:09:36.774035 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:36.773996    2572 generic.go:358] "Generic (PLEG): container finished" podID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerID="dc26945181d0268adebf19a43e2a14bec6174a77432c393adc7ddbf18ecd4fdb" exitCode=137
May 25 07:09:36.774363 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:36.774064    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" event={"ID":"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4","Type":"ContainerDied","Data":"dc26945181d0268adebf19a43e2a14bec6174a77432c393adc7ddbf18ecd4fdb"}
May 25 07:09:37.362675 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.362652    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:09:37.411455 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.411427    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-ui-tls\") pod \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") "
May 25 07:09:37.411637 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.411482    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-registry-tls\") pod \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") "
May 25 07:09:37.411637 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.411514    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-feast-data\") pod \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") "
May 25 07:09:37.411637 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.411547    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cp24h\" (UniqueName: \"kubernetes.io/projected/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-kube-api-access-cp24h\") pod \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") "
May 25 07:09:37.411637 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.411567    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-online-tls\") pod \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") "
May 25 07:09:37.411637 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.411589    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-offline-tls\") pod \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\" (UID: \"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4\") "
May 25 07:09:37.411998 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.411972    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" (UID: "debbb6fc-d5d5-4d80-9ea9-94a7332f17c4"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 07:09:37.412370 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.412336    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-feast-data" (OuterVolumeSpecName: "feast-data") pod "debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" (UID: "debbb6fc-d5d5-4d80-9ea9-94a7332f17c4"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 07:09:37.413808 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.413770    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-kube-api-access-cp24h" (OuterVolumeSpecName: "kube-api-access-cp24h") pod "debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" (UID: "debbb6fc-d5d5-4d80-9ea9-94a7332f17c4"). InnerVolumeSpecName "kube-api-access-cp24h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 07:09:37.413939 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.413915    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-online-tls" (OuterVolumeSpecName: "online-tls") pod "debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" (UID: "debbb6fc-d5d5-4d80-9ea9-94a7332f17c4"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:09:37.414173 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.414147    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" (UID: "debbb6fc-d5d5-4d80-9ea9-94a7332f17c4"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:09:37.414266 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.414177    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" (UID: "debbb6fc-d5d5-4d80-9ea9-94a7332f17c4"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:09:37.498547 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.498521    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:09:37.513141 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.513116    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cp24h\" (UniqueName: \"kubernetes.io/projected/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-kube-api-access-cp24h\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:09:37.513141 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.513143    2572 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-online-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:09:37.513285 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.513153    2572 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-offline-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:09:37.513285 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.513161    2572 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-ui-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:09:37.513285 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.513169    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:09:37.513285 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.513177    2572 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4-feast-data\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:09:37.614371 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.614337    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-registry-tls\") pod \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") "
May 25 07:09:37.614371 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.614379    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-offline-tls\") pod \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") "
May 25 07:09:37.614598 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.614405    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-feast-data\") pod \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") "
May 25 07:09:37.614598 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.614438    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-llvzd\" (UniqueName: \"kubernetes.io/projected/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-kube-api-access-llvzd\") pod \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") "
May 25 07:09:37.614598 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.614461    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-ui-tls\") pod \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") "
May 25 07:09:37.614598 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.614482    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-online-tls\") pod \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\" (UID: \"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2\") "
May 25 07:09:37.615021 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.614989    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-feast-data" (OuterVolumeSpecName: "feast-data") pod "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" (UID: "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 07:09:37.616673 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.616639    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" (UID: "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:09:37.616758 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.616686    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-online-tls" (OuterVolumeSpecName: "online-tls") pod "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" (UID: "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:09:37.616758 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.616709    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" (UID: "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:09:37.617026 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.617004    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" (UID: "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:09:37.617026 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.617012    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-kube-api-access-llvzd" (OuterVolumeSpecName: "kube-api-access-llvzd") pod "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" (UID: "83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2"). InnerVolumeSpecName "kube-api-access-llvzd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 07:09:37.715360 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.715318    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:09:37.715360 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.715349    2572 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-offline-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:09:37.715360 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.715357    2572 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-feast-data\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:09:37.715360 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.715367    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-llvzd\" (UniqueName: \"kubernetes.io/projected/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-kube-api-access-llvzd\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:09:37.715703 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.715376    2572 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-ui-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:09:37.715703 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.715386    2572 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2-online-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:09:37.779695 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.779611    2572 generic.go:358] "Generic (PLEG): container finished" podID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerID="383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94" exitCode=137
May 25 07:09:37.779695 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.779638    2572 generic.go:358] "Generic (PLEG): container finished" podID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerID="e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e" exitCode=137
May 25 07:09:37.780188 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.779710    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerDied","Data":"383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94"}
May 25 07:09:37.780188 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.779735    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerDied","Data":"e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e"}
May 25 07:09:37.780188 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.779741    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"
May 25 07:09:37.780188 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.779762    2572 scope.go:117] "RemoveContainer" containerID="5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2"
May 25 07:09:37.780188 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.779748    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz" event={"ID":"83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2","Type":"ContainerDied","Data":"e8bf1cffb1874f0111d5ab7a9e5acb7f15c293160d449842ac67062937abc57d"}
May 25 07:09:37.782138 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.782116    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb" event={"ID":"debbb6fc-d5d5-4d80-9ea9-94a7332f17c4","Type":"ContainerDied","Data":"e394f9484a9d2d0260bf581010242129adfaed4ac4a942dc40fd9f1d74d2806c"}
May 25 07:09:37.782239 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.782165    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"
May 25 07:09:37.789944 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.789926    2572 scope.go:117] "RemoveContainer" containerID="383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94"
May 25 07:09:37.797294 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.797279    2572 scope.go:117] "RemoveContainer" containerID="00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75"
May 25 07:09:37.803368 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.803348    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"]
May 25 07:09:37.804491 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.804474    2572 scope.go:117] "RemoveContainer" containerID="e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e"
May 25 07:09:37.808700 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.808669    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lmgqb"]
May 25 07:09:37.812278 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.812260    2572 scope.go:117] "RemoveContainer" containerID="f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea"
May 25 07:09:37.817662 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.817642    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"]
May 25 07:09:37.819706 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.819689    2572 scope.go:117] "RemoveContainer" containerID="c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8"
May 25 07:09:37.820065 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.820048    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-gnprz"]
May 25 07:09:37.826516 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.826497    2572 scope.go:117] "RemoveContainer" containerID="5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2"
May 25 07:09:37.826791 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:09:37.826769    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2\": container with ID starting with 5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2 not found: ID does not exist" containerID="5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2"
May 25 07:09:37.826970 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.826797    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2"} err="failed to get container status \"5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2\": rpc error: code = NotFound desc = could not find container \"5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2\": container with ID starting with 5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2 not found: ID does not exist"
May 25 07:09:37.826970 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.826819    2572 scope.go:117] "RemoveContainer" containerID="383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94"
May 25 07:09:37.827083 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:09:37.827065    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94\": container with ID starting with 383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94 not found: ID does not exist" containerID="383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94"
May 25 07:09:37.827121 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.827089    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94"} err="failed to get container status \"383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94\": rpc error: code = NotFound desc = could not find container \"383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94\": container with ID starting with 383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94 not found: ID does not exist"
May 25 07:09:37.827121 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.827105    2572 scope.go:117] "RemoveContainer" containerID="00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75"
May 25 07:09:37.827297 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:09:37.827279    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75\": container with ID starting with 00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75 not found: ID does not exist" containerID="00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75"
May 25 07:09:37.827335 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.827303    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75"} err="failed to get container status \"00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75\": rpc error: code = NotFound desc = could not find container \"00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75\": container with ID starting with 00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75 not found: ID does not exist"
May 25 07:09:37.827335 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.827318    2572 scope.go:117] "RemoveContainer" containerID="e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e"
May 25 07:09:37.827499 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:09:37.827481    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e\": container with ID starting with e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e not found: ID does not exist" containerID="e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e"
May 25 07:09:37.827537 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.827508    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e"} err="failed to get container status \"e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e\": rpc error: code = NotFound desc = could not find container \"e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e\": container with ID starting with e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e not found: ID does not exist"
May 25 07:09:37.827537 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.827531    2572 scope.go:117] "RemoveContainer" containerID="f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea"
May 25 07:09:37.827721 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:09:37.827704    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea\": container with ID starting with f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea not found: ID does not exist" containerID="f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea"
May 25 07:09:37.827786 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.827730    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea"} err="failed to get container status \"f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea\": rpc error: code = NotFound desc = could not find container \"f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea\": container with ID starting with f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea not found: ID does not exist"
May 25 07:09:37.827786 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.827755    2572 scope.go:117] "RemoveContainer" containerID="c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8"
May 25 07:09:37.828000 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:09:37.827979    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8\": container with ID starting with c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8 not found: ID does not exist" containerID="c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8"
May 25 07:09:37.828051 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.828005    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8"} err="failed to get container status \"c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8\": rpc error: code = NotFound desc = could not find container \"c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8\": container with ID starting with c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8 not found: ID does not exist"
May 25 07:09:37.828051 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.828018    2572 scope.go:117] "RemoveContainer" containerID="5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2"
May 25 07:09:37.828246 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.828230    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2"} err="failed to get container status \"5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2\": rpc error: code = NotFound desc = could not find container \"5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2\": container with ID starting with 5860fcb9f4e2ea02b3522c612f836f189ca72a487193ffab9b2337df23b70ec2 not found: ID does not exist"
May 25 07:09:37.828315 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.828246    2572 scope.go:117] "RemoveContainer" containerID="383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94"
May 25 07:09:37.828474 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.828453    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94"} err="failed to get container status \"383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94\": rpc error: code = NotFound desc = could not find container \"383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94\": container with ID starting with 383b3908475fb9538fb4ff9c7545d859625b2428a173c1e36d864d0872cc6c94 not found: ID does not exist"
May 25 07:09:37.828532 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.828474    2572 scope.go:117] "RemoveContainer" containerID="00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75"
May 25 07:09:37.828713 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.828692    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75"} err="failed to get container status \"00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75\": rpc error: code = NotFound desc = could not find container \"00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75\": container with ID starting with 00320a5b2f3e093cc090d495a208e8dde2f1c7d0a2386ccb26bccccdc0f02d75 not found: ID does not exist"
May 25 07:09:37.828759 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.828714    2572 scope.go:117] "RemoveContainer" containerID="e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e"
May 25 07:09:37.828945 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.828926    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e"} err="failed to get container status \"e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e\": rpc error: code = NotFound desc = could not find container \"e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e\": container with ID starting with e4c70b1b0baa902cd83b65cf26fc9532aa584a81a98754b3f5e59b1dfdc0a25e not found: ID does not exist"
May 25 07:09:37.829002 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.828948    2572 scope.go:117] "RemoveContainer" containerID="f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea"
May 25 07:09:37.829167 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.829148    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea"} err="failed to get container status \"f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea\": rpc error: code = NotFound desc = could not find container \"f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea\": container with ID starting with f8503e88e1123dc36f8a5473c7299dd188d6f0602f796e1d803e18c42e5c75ea not found: ID does not exist"
May 25 07:09:37.829214 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.829168    2572 scope.go:117] "RemoveContainer" containerID="c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8"
May 25 07:09:37.829353 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.829333    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8"} err="failed to get container status \"c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8\": rpc error: code = NotFound desc = could not find container \"c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8\": container with ID starting with c4602ea27d7f87546d4b1cff36091da81b074cf7137edc05c6f7ea26e40547e8 not found: ID does not exist"
May 25 07:09:37.829395 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.829353    2572 scope.go:117] "RemoveContainer" containerID="8d9db5e3b2ac35648c13882fc752ff73b020c310318242f7813442c8ca19eacd"
May 25 07:09:37.836317 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.836300    2572 scope.go:117] "RemoveContainer" containerID="dc26945181d0268adebf19a43e2a14bec6174a77432c393adc7ddbf18ecd4fdb"
May 25 07:09:37.843081 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.843065    2572 scope.go:117] "RemoveContainer" containerID="15bb892988e41f4bfdeb702e7d1b9e2601e3e57cdc946005f2c9d2b0fe6ec3f1"
May 25 07:09:37.849514 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.849498    2572 scope.go:117] "RemoveContainer" containerID="db70b93e094466a5be17d0b2a23983d7fecaf21f0a9e670463a16d127b1d0aff"
May 25 07:09:37.856020 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:37.856004    2572 scope.go:117] "RemoveContainer" containerID="10d8ebabfa184eea72bf20d8f8a2572c4f5307c52f5121901197aaf1993f47dc"
May 25 07:09:38.281697 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:38.281668    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" path="/var/lib/kubelet/pods/83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2/volumes"
May 25 07:09:38.282274 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:09:38.282260    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" path="/var/lib/kubelet/pods/debbb6fc-d5d5-4d80-9ea9-94a7332f17c4/volumes"
May 25 07:10:15.789674 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.789614    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"]
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.789987    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="feast-apply"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790001    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="feast-apply"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790011    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="feast-init"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790020    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="feast-init"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790028    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="offline"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790034    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="offline"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790043    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="ui"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790048    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="ui"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790056    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="offline"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790061    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="offline"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790068    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="online"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790075    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="online"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790081    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="feast-init"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790086    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="feast-init"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790095    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="online"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790103    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="online"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790113    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="feast-apply"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790118    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="feast-apply"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790123    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="registry"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790128    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="registry"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790138    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="ui"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790144    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="ui"
May 25 07:10:15.790187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790194    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="registry"
May 25 07:10:15.790909 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790203    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="online"
May 25 07:10:15.790909 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790210    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="ui"
May 25 07:10:15.790909 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790217    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="online"
May 25 07:10:15.790909 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790224    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="83dfdf28-aae7-4406-b0e5-3e7a0e85c4f2" containerName="offline"
May 25 07:10:15.790909 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790230    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="offline"
May 25 07:10:15.790909 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.790236    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="debbb6fc-d5d5-4d80-9ea9-94a7332f17c4" containerName="ui"
May 25 07:10:15.794905 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.794886    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:15.796807 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.796773    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 25 07:10:15.796957 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.796824    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-mt4rr\""
May 25 07:10:15.796957 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.796865    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 25 07:10:15.797057 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.797028    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 25 07:10:15.797095 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.797053    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 25 07:10:15.797095 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.797059    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 25 07:10:15.797095 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.797059    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 25 07:10:15.804145 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.804125    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"]
May 25 07:10:15.928102 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.928069    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:15.928303 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.928117    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:15.928303 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.928170    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:15.928303 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.928217    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:15.928303 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.928247    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-56v5s\" (UniqueName: \"kubernetes.io/projected/57497e1e-7488-44fd-92e1-52baab49ffd8-kube-api-access-56v5s\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:15.928483 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:15.928315    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/57497e1e-7488-44fd-92e1-52baab49ffd8-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.028725 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.028682    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.028725 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.028722    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.029022 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.028745    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.029022 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.028772    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-56v5s\" (UniqueName: \"kubernetes.io/projected/57497e1e-7488-44fd-92e1-52baab49ffd8-kube-api-access-56v5s\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.029022 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.028804    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/57497e1e-7488-44fd-92e1-52baab49ffd8-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.029022 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.028884    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.029297 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.029260    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/57497e1e-7488-44fd-92e1-52baab49ffd8-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.031207 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.031176    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.031331 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.031302    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.031331 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.031306    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.031460 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.031439    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.035718 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.035695    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-56v5s\" (UniqueName: \"kubernetes.io/projected/57497e1e-7488-44fd-92e1-52baab49ffd8-kube-api-access-56v5s\") pod \"feast-simple-feast-setup-664fcfcc58-nwxjq\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.110437 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.110402    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:16.242227 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.242192    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"]
May 25 07:10:16.245609 ip-10-0-128-6 kubenswrapper[2572]: W0525 07:10:16.245571    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57497e1e_7488_44fd_92e1_52baab49ffd8.slice/crio-a85c36af9a2ddcabca330296bd88fa76a048ba3f6cd2989475108e30dde4113b WatchSource:0}: Error finding container a85c36af9a2ddcabca330296bd88fa76a048ba3f6cd2989475108e30dde4113b: Status 404 returned error can't find the container with id a85c36af9a2ddcabca330296bd88fa76a048ba3f6cd2989475108e30dde4113b
May 25 07:10:16.247319 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.247303    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 07:10:16.897399 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.897356    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerStarted","Data":"669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8"}
May 25 07:10:16.897399 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:16.897403    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerStarted","Data":"a85c36af9a2ddcabca330296bd88fa76a048ba3f6cd2989475108e30dde4113b"}
May 25 07:10:20.968357 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:10:20.968322    2572 cadvisor_stats_provider.go:525] "Partial failure issuing cadvisor.ContainerInfoV2" err="partial failures: [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57497e1e_7488_44fd_92e1_52baab49ffd8.slice/crio-669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8.scope\": RecentStats: unable to find data in memory cache], [\"/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod57497e1e_7488_44fd_92e1_52baab49ffd8.slice/crio-conmon-669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8.scope\": RecentStats: unable to find data in memory cache]"
May 25 07:10:21.913520 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:21.913487    2572 generic.go:358] "Generic (PLEG): container finished" podID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerID="669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8" exitCode=0
May 25 07:10:21.913786 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:21.913565    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerDied","Data":"669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8"}
May 25 07:10:22.918677 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:22.918638    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerStarted","Data":"af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f"}
May 25 07:10:27.935239 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:27.935206    2572 generic.go:358] "Generic (PLEG): container finished" podID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerID="af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f" exitCode=0
May 25 07:10:27.935613 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:27.935278    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerDied","Data":"af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f"}
May 25 07:10:28.942832 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:28.942788    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerStarted","Data":"8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12"}
May 25 07:10:28.943331 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:28.942870    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerStarted","Data":"793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639"}
May 25 07:10:28.943331 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:28.942890    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerStarted","Data":"cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209"}
May 25 07:10:28.943331 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:28.942904    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerStarted","Data":"133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036"}
May 25 07:10:28.964187 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:28.964128    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podStartSLOduration=13.964108569 podStartE2EDuration="13.964108569s" podCreationTimestamp="2026-05-25 07:10:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 07:10:28.963441425 +0000 UTC m=+1467.199627817" watchObservedRunningTime="2026-05-25 07:10:28.964108569 +0000 UTC m=+1467.200294963"
May 25 07:10:31.111141 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:31.111097    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:31.111141 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:31.111150    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:31.111682 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:31.111165    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:31.111682 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:31.111180    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:31.113225 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:31.113180    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 25 07:10:31.113377 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:31.113213    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.37:6571: connect: connection refused"
May 25 07:10:31.113377 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:31.113282    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 25 07:10:31.113550 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:31.113515    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.37:8816: connect: connection refused"
May 25 07:10:34.111559 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:34.111523    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:34.112047 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:34.111703    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:34.112131 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:34.112072    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:34.112131 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:34.112104    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:34.112131 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:34.112117    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:34.112131 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:34.112130    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:34.112432 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:34.112411    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:34.112616 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:34.112599    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:34.112683 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:34.112646    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:34.116373 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:34.116353    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:34.963763 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:34.963736    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:10:34.970323 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:10:34.970293    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:11:02.301387 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:02.301358    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 07:11:02.305193 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:02.305171    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 07:11:37.520198 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:37.520110    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"]
May 25 07:11:37.520772 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:37.520621    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="registry" containerID="cri-o://133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036" gracePeriod=30
May 25 07:11:37.520772 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:37.520710    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="offline" containerID="cri-o://793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639" gracePeriod=30
May 25 07:11:37.520772 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:37.520761    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="online" containerID="cri-o://cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209" gracePeriod=30
May 25 07:11:37.520971 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:37.520714    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="ui" containerID="cri-o://8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12" gracePeriod=30
May 25 07:11:37.901114 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:37.901080    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"]
May 25 07:11:37.904785 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:37.904759    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:37.906765 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:37.906741    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-hz2sl\""
May 25 07:11:37.928382 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:37.928343    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"]
May 25 07:11:38.001818 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.001773    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.002015 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.001824    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9bbb034c-1823-4ffe-851b-1b2723236830-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.002015 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.001953    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p8hvw\" (UniqueName: \"kubernetes.io/projected/9bbb034c-1823-4ffe-851b-1b2723236830-kube-api-access-p8hvw\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.002015 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.002000    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.002161 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.002031    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.002161 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.002060    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.102927 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.102881    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.103098 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.102936    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9bbb034c-1823-4ffe-851b-1b2723236830-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.103098 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.102985    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p8hvw\" (UniqueName: \"kubernetes.io/projected/9bbb034c-1823-4ffe-851b-1b2723236830-kube-api-access-p8hvw\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.103098 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.103019    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.103266 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.103143    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.103266 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.103197    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.103453 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.103425    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9bbb034c-1823-4ffe-851b-1b2723236830-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.105725 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.105698    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.105725 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.105720    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.105950 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.105812    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.106012 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.105978    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.109457 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.109422    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p8hvw\" (UniqueName: \"kubernetes.io/projected/9bbb034c-1823-4ffe-851b-1b2723236830-kube-api-access-p8hvw\") pod \"feast-simple-feast-setup-664fcfcc58-slsrm\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.142673 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.142643    2572 generic.go:358] "Generic (PLEG): container finished" podID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerID="8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12" exitCode=0
May 25 07:11:38.142837 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.142717    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerDied","Data":"8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12"}
May 25 07:11:38.214217 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.214177    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:38.343066 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:38.342883    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"]
May 25 07:11:38.345138 ip-10-0-128-6 kubenswrapper[2572]: W0525 07:11:38.345115    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9bbb034c_1823_4ffe_851b_1b2723236830.slice/crio-af7dc785724d56022376887168d7092162b9dd7ee9ed3d957f25781fab613d66 WatchSource:0}: Error finding container af7dc785724d56022376887168d7092162b9dd7ee9ed3d957f25781fab613d66: Status 404 returned error can't find the container with id af7dc785724d56022376887168d7092162b9dd7ee9ed3d957f25781fab613d66
May 25 07:11:39.147222 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:39.147180    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerStarted","Data":"0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2"}
May 25 07:11:39.147222 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:39.147228    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerStarted","Data":"af7dc785724d56022376887168d7092162b9dd7ee9ed3d957f25781fab613d66"}
May 25 07:11:39.149900 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:39.149874    2572 generic.go:358] "Generic (PLEG): container finished" podID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerID="cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209" exitCode=0
May 25 07:11:39.150041 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:39.149953    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerDied","Data":"cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209"}
May 25 07:11:43.163690 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:43.163651    2572 generic.go:358] "Generic (PLEG): container finished" podID="9bbb034c-1823-4ffe-851b-1b2723236830" containerID="0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2" exitCode=0
May 25 07:11:43.164081 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:43.163724    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerDied","Data":"0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2"}
May 25 07:11:44.113088 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:44.113038    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 25 07:11:44.167798 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:44.167764    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerStarted","Data":"03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9"}
May 25 07:11:44.964246 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:44.964195    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 25 07:11:49.183744 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:49.183712    2572 generic.go:358] "Generic (PLEG): container finished" podID="9bbb034c-1823-4ffe-851b-1b2723236830" containerID="03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9" exitCode=0
May 25 07:11:49.184126 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:49.183782    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerDied","Data":"03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9"}
May 25 07:11:50.191549 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.191506    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerStarted","Data":"312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9"}
May 25 07:11:50.192066 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.191565    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerStarted","Data":"992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1"}
May 25 07:11:50.192066 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.191582    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerStarted","Data":"3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84"}
May 25 07:11:50.192066 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.191598    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerStarted","Data":"9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c"}
May 25 07:11:50.213544 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.213478    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podStartSLOduration=13.213457744 podStartE2EDuration="13.213457744s" podCreationTimestamp="2026-05-25 07:11:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 07:11:50.211005499 +0000 UTC m=+1548.447191891" watchObservedRunningTime="2026-05-25 07:11:50.213457744 +0000 UTC m=+1548.449644138"
May 25 07:11:50.214854 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.214802    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:50.215077 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.215058    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:50.215208 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.215187    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:50.215208 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.215210    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:50.216700 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.216658    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.38:8816: connect: connection refused"
May 25 07:11:50.216700 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.216674    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.38:6571: connect: connection refused"
May 25 07:11:50.216978 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.216680    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 25 07:11:50.216978 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:50.216739    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 25 07:11:53.215306 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:53.215234    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.38:6571: connect: connection refused"
May 25 07:11:53.215306 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:53.215278    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 25 07:11:53.218075 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:53.215235    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.38:8816: connect: connection refused"
May 25 07:11:53.218075 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:53.215242    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 25 07:11:54.113361 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:54.113312    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 25 07:11:54.964559 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:54.964510    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 25 07:11:56.216030 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:56.215994    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:56.216507 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:56.216142    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:56.216586 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:56.216503    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:56.216586 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:56.216531    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:56.216586 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:56.216547    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:56.216586 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:56.216559    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:56.216893 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:56.216874    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:56.216979 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:56.216933    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:56.217021 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:56.216984    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:56.220506 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:56.220489    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:57.216857 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:57.216817    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:11:57.219832 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:11:57.219809    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:12:04.112884 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:04.112795    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 25 07:12:04.113286 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:04.112978    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:12:04.964258 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:04.964221    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 25 07:12:04.964448 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:04.964335    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:12:08.172582 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.172558    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:12:08.248938 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.248901    2572 generic.go:358] "Generic (PLEG): container finished" podID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerID="793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639" exitCode=137
May 25 07:12:08.248938 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.248934    2572 generic.go:358] "Generic (PLEG): container finished" podID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerID="133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036" exitCode=137
May 25 07:12:08.249244 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.248982    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"
May 25 07:12:08.249244 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.248995    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerDied","Data":"793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639"}
May 25 07:12:08.249244 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.249038    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerDied","Data":"133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036"}
May 25 07:12:08.249244 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.249050    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq" event={"ID":"57497e1e-7488-44fd-92e1-52baab49ffd8","Type":"ContainerDied","Data":"a85c36af9a2ddcabca330296bd88fa76a048ba3f6cd2989475108e30dde4113b"}
May 25 07:12:08.249244 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.249066    2572 scope.go:117] "RemoveContainer" containerID="8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12"
May 25 07:12:08.257301 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.257284    2572 scope.go:117] "RemoveContainer" containerID="793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639"
May 25 07:12:08.264335 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.264318    2572 scope.go:117] "RemoveContainer" containerID="cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209"
May 25 07:12:08.264761 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.264740    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-offline-tls\") pod \"57497e1e-7488-44fd-92e1-52baab49ffd8\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") "
May 25 07:12:08.264988 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.264780    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-registry-tls\") pod \"57497e1e-7488-44fd-92e1-52baab49ffd8\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") "
May 25 07:12:08.264988 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.264814    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/57497e1e-7488-44fd-92e1-52baab49ffd8-feast-data\") pod \"57497e1e-7488-44fd-92e1-52baab49ffd8\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") "
May 25 07:12:08.264988 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.264881    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-56v5s\" (UniqueName: \"kubernetes.io/projected/57497e1e-7488-44fd-92e1-52baab49ffd8-kube-api-access-56v5s\") pod \"57497e1e-7488-44fd-92e1-52baab49ffd8\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") "
May 25 07:12:08.264988 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.264931    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-online-tls\") pod \"57497e1e-7488-44fd-92e1-52baab49ffd8\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") "
May 25 07:12:08.264988 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.264953    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-ui-tls\") pod \"57497e1e-7488-44fd-92e1-52baab49ffd8\" (UID: \"57497e1e-7488-44fd-92e1-52baab49ffd8\") "
May 25 07:12:08.265801 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.265758    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/57497e1e-7488-44fd-92e1-52baab49ffd8-feast-data" (OuterVolumeSpecName: "feast-data") pod "57497e1e-7488-44fd-92e1-52baab49ffd8" (UID: "57497e1e-7488-44fd-92e1-52baab49ffd8"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 07:12:08.267343 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.267298    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "57497e1e-7488-44fd-92e1-52baab49ffd8" (UID: "57497e1e-7488-44fd-92e1-52baab49ffd8"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:12:08.267753 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.267720    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-online-tls" (OuterVolumeSpecName: "online-tls") pod "57497e1e-7488-44fd-92e1-52baab49ffd8" (UID: "57497e1e-7488-44fd-92e1-52baab49ffd8"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:12:08.267876 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.267770    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/57497e1e-7488-44fd-92e1-52baab49ffd8-kube-api-access-56v5s" (OuterVolumeSpecName: "kube-api-access-56v5s") pod "57497e1e-7488-44fd-92e1-52baab49ffd8" (UID: "57497e1e-7488-44fd-92e1-52baab49ffd8"). InnerVolumeSpecName "kube-api-access-56v5s". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 07:12:08.267876 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.267857    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "57497e1e-7488-44fd-92e1-52baab49ffd8" (UID: "57497e1e-7488-44fd-92e1-52baab49ffd8"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:12:08.267876 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.267866    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "57497e1e-7488-44fd-92e1-52baab49ffd8" (UID: "57497e1e-7488-44fd-92e1-52baab49ffd8"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:12:08.278441 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.278419    2572 scope.go:117] "RemoveContainer" containerID="133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036"
May 25 07:12:08.285825 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.285803    2572 scope.go:117] "RemoveContainer" containerID="af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f"
May 25 07:12:08.293440 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.293420    2572 scope.go:117] "RemoveContainer" containerID="669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8"
May 25 07:12:08.301086 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.301065    2572 scope.go:117] "RemoveContainer" containerID="8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12"
May 25 07:12:08.301349 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:12:08.301315    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12\": container with ID starting with 8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12 not found: ID does not exist" containerID="8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12"
May 25 07:12:08.301395 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.301356    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12"} err="failed to get container status \"8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12\": rpc error: code = NotFound desc = could not find container \"8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12\": container with ID starting with 8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12 not found: ID does not exist"
May 25 07:12:08.301395 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.301375    2572 scope.go:117] "RemoveContainer" containerID="793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639"
May 25 07:12:08.301589 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:12:08.301572    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639\": container with ID starting with 793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639 not found: ID does not exist" containerID="793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639"
May 25 07:12:08.301633 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.301599    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639"} err="failed to get container status \"793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639\": rpc error: code = NotFound desc = could not find container \"793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639\": container with ID starting with 793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639 not found: ID does not exist"
May 25 07:12:08.301633 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.301620    2572 scope.go:117] "RemoveContainer" containerID="cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209"
May 25 07:12:08.301871 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:12:08.301830    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209\": container with ID starting with cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209 not found: ID does not exist" containerID="cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209"
May 25 07:12:08.301935 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.301875    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209"} err="failed to get container status \"cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209\": rpc error: code = NotFound desc = could not find container \"cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209\": container with ID starting with cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209 not found: ID does not exist"
May 25 07:12:08.301935 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.301890    2572 scope.go:117] "RemoveContainer" containerID="133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036"
May 25 07:12:08.302088 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:12:08.302071    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036\": container with ID starting with 133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036 not found: ID does not exist" containerID="133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036"
May 25 07:12:08.302127 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.302094    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036"} err="failed to get container status \"133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036\": rpc error: code = NotFound desc = could not find container \"133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036\": container with ID starting with 133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036 not found: ID does not exist"
May 25 07:12:08.302127 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.302106    2572 scope.go:117] "RemoveContainer" containerID="af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f"
May 25 07:12:08.302320 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:12:08.302300    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f\": container with ID starting with af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f not found: ID does not exist" containerID="af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f"
May 25 07:12:08.302375 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.302322    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f"} err="failed to get container status \"af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f\": rpc error: code = NotFound desc = could not find container \"af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f\": container with ID starting with af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f not found: ID does not exist"
May 25 07:12:08.302375 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.302336    2572 scope.go:117] "RemoveContainer" containerID="669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8"
May 25 07:12:08.302552 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:12:08.302527    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8\": container with ID starting with 669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8 not found: ID does not exist" containerID="669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8"
May 25 07:12:08.302595 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.302556    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8"} err="failed to get container status \"669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8\": rpc error: code = NotFound desc = could not find container \"669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8\": container with ID starting with 669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8 not found: ID does not exist"
May 25 07:12:08.302595 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.302570    2572 scope.go:117] "RemoveContainer" containerID="8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12"
May 25 07:12:08.302777 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.302760    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12"} err="failed to get container status \"8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12\": rpc error: code = NotFound desc = could not find container \"8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12\": container with ID starting with 8442e4c191686894058d4073cb8a916733e9caf468091a216708e1e469cd7b12 not found: ID does not exist"
May 25 07:12:08.302817 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.302777    2572 scope.go:117] "RemoveContainer" containerID="793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639"
May 25 07:12:08.303054 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.303036    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639"} err="failed to get container status \"793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639\": rpc error: code = NotFound desc = could not find container \"793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639\": container with ID starting with 793d52427a5395deb6283a2d77733f828f62265691add4389a237da06ccd2639 not found: ID does not exist"
May 25 07:12:08.303112 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.303064    2572 scope.go:117] "RemoveContainer" containerID="cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209"
May 25 07:12:08.303309 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.303285    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209"} err="failed to get container status \"cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209\": rpc error: code = NotFound desc = could not find container \"cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209\": container with ID starting with cc8bde799a230ab362ccb9930e6940c202c674829ec31994831312d56d8d4209 not found: ID does not exist"
May 25 07:12:08.303309 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.303305    2572 scope.go:117] "RemoveContainer" containerID="133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036"
May 25 07:12:08.303519 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.303502    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036"} err="failed to get container status \"133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036\": rpc error: code = NotFound desc = could not find container \"133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036\": container with ID starting with 133889e97f41c0a570473b27eb6e01a814eb6ab15ef83e18f35725a96bd4c036 not found: ID does not exist"
May 25 07:12:08.303569 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.303520    2572 scope.go:117] "RemoveContainer" containerID="af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f"
May 25 07:12:08.303738 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.303721    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f"} err="failed to get container status \"af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f\": rpc error: code = NotFound desc = could not find container \"af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f\": container with ID starting with af44ced5991f04a158fc6ccdde84065f01bef418dec75a1ad3ed6f133ebf602f not found: ID does not exist"
May 25 07:12:08.303782 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.303739    2572 scope.go:117] "RemoveContainer" containerID="669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8"
May 25 07:12:08.303953 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.303936    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8"} err="failed to get container status \"669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8\": rpc error: code = NotFound desc = could not find container \"669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8\": container with ID starting with 669da22e6b85f0811099adfed78a80c3100a011917b0253ecfd6461d9dad44a8 not found: ID does not exist"
May 25 07:12:08.366408 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.366372    2572 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-offline-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:12:08.366408 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.366401    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:12:08.366408 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.366410    2572 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/57497e1e-7488-44fd-92e1-52baab49ffd8-feast-data\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:12:08.366408 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.366421    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-56v5s\" (UniqueName: \"kubernetes.io/projected/57497e1e-7488-44fd-92e1-52baab49ffd8-kube-api-access-56v5s\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:12:08.366689 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.366431    2572 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-online-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:12:08.366689 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.366441    2572 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/57497e1e-7488-44fd-92e1-52baab49ffd8-ui-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:12:08.565157 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.565132    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"]
May 25 07:12:08.573825 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:08.573799    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-nwxjq"]
May 25 07:12:10.282608 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:10.282573    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" path="/var/lib/kubelet/pods/57497e1e-7488-44fd-92e1-52baab49ffd8/volumes"
May 25 07:12:59.640533 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640444    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"]
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640784    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="registry"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640797    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="registry"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640812    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="feast-init"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640818    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="feast-init"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640828    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="online"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640834    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="online"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640873    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="offline"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640881    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="offline"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640887    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="ui"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640895    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="ui"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640902    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="feast-apply"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640908    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="feast-apply"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640965    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="offline"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640974    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="online"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640982    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="registry"
May 25 07:12:59.641135 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.640988    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="57497e1e-7488-44fd-92e1-52baab49ffd8" containerName="ui"
May 25 07:12:59.644594 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.644577    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.646329 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.646305    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 25 07:12:59.646483 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.646464    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 25 07:12:59.646572 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.646486    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-5c95w\""
May 25 07:12:59.646572 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.646553    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 25 07:12:59.646693 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.646556    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 25 07:12:59.646740 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.646697    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 25 07:12:59.646827 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.646810    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 25 07:12:59.654234 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.654213    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"]
May 25 07:12:59.697561 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.697531    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.697729 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.697575    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.697729 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.697595    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.697729 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.697678    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-sxgjh\" (UniqueName: \"kubernetes.io/projected/8b45503d-f54c-4359-a750-ab1f3f82c626-kube-api-access-sxgjh\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.697729 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.697706    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/8b45503d-f54c-4359-a750-ab1f3f82c626-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.697895 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.697759    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8b45503d-f54c-4359-a750-ab1f3f82c626-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.798671 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.798642    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.798833 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.798696    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.798833 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.798803    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.798959 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.798868    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-sxgjh\" (UniqueName: \"kubernetes.io/projected/8b45503d-f54c-4359-a750-ab1f3f82c626-kube-api-access-sxgjh\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.798959 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.798893    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/8b45503d-f54c-4359-a750-ab1f3f82c626-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.798959 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.798937    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8b45503d-f54c-4359-a750-ab1f3f82c626-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.799347 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.799320    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8b45503d-f54c-4359-a750-ab1f3f82c626-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.799598 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.799579    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/8b45503d-f54c-4359-a750-ab1f3f82c626-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.801329 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.801300    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.801329 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.801324    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.801481 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.801347    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.805748 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.805726    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-sxgjh\" (UniqueName: \"kubernetes.io/projected/8b45503d-f54c-4359-a750-ab1f3f82c626-kube-api-access-sxgjh\") pod \"feast-simple-feast-remote-setup-5cff94746b-8l9d9\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:12:59.955071 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:12:59.954979    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:13:00.085576 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:00.085543    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"]
May 25 07:13:00.088618 ip-10-0-128-6 kubenswrapper[2572]: W0525 07:13:00.088589    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8b45503d_f54c_4359_a750_ab1f3f82c626.slice/crio-76545ef9b99e902c5c5c9a66888302c02b2a6ba0172ef98d781dcc3df76931c2 WatchSource:0}: Error finding container 76545ef9b99e902c5c5c9a66888302c02b2a6ba0172ef98d781dcc3df76931c2: Status 404 returned error can't find the container with id 76545ef9b99e902c5c5c9a66888302c02b2a6ba0172ef98d781dcc3df76931c2
May 25 07:13:00.396518 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:00.395714    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" event={"ID":"8b45503d-f54c-4359-a750-ab1f3f82c626","Type":"ContainerStarted","Data":"d1d53c4d18993efa032f8f50a892698d6762ade878c28bc161bf93c782469a31"}
May 25 07:13:00.396518 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:00.395759    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" event={"ID":"8b45503d-f54c-4359-a750-ab1f3f82c626","Type":"ContainerStarted","Data":"76545ef9b99e902c5c5c9a66888302c02b2a6ba0172ef98d781dcc3df76931c2"}
May 25 07:13:05.412636 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:05.412603    2572 generic.go:358] "Generic (PLEG): container finished" podID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerID="d1d53c4d18993efa032f8f50a892698d6762ade878c28bc161bf93c782469a31" exitCode=0
May 25 07:13:05.413049 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:05.412669    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" event={"ID":"8b45503d-f54c-4359-a750-ab1f3f82c626","Type":"ContainerDied","Data":"d1d53c4d18993efa032f8f50a892698d6762ade878c28bc161bf93c782469a31"}
May 25 07:13:06.418270 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:06.418231    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" event={"ID":"8b45503d-f54c-4359-a750-ab1f3f82c626","Type":"ContainerStarted","Data":"2a1af6028c85b3f4f553cc5a458af0f9f3d9f1f2eeb5fa7223125e37ba6f7a4d"}
May 25 07:13:11.436472 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:11.436380    2572 generic.go:358] "Generic (PLEG): container finished" podID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerID="2a1af6028c85b3f4f553cc5a458af0f9f3d9f1f2eeb5fa7223125e37ba6f7a4d" exitCode=0
May 25 07:13:11.436472 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:11.436456    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" event={"ID":"8b45503d-f54c-4359-a750-ab1f3f82c626","Type":"ContainerDied","Data":"2a1af6028c85b3f4f553cc5a458af0f9f3d9f1f2eeb5fa7223125e37ba6f7a4d"}
May 25 07:13:12.443497 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:12.443456    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" event={"ID":"8b45503d-f54c-4359-a750-ab1f3f82c626","Type":"ContainerStarted","Data":"209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441"}
May 25 07:13:12.443902 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:12.443511    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" event={"ID":"8b45503d-f54c-4359-a750-ab1f3f82c626","Type":"ContainerStarted","Data":"3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185"}
May 25 07:13:12.443902 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:12.443528    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" event={"ID":"8b45503d-f54c-4359-a750-ab1f3f82c626","Type":"ContainerStarted","Data":"8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755"}
May 25 07:13:12.464858 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:12.464772    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podStartSLOduration=13.464752776 podStartE2EDuration="13.464752776s" podCreationTimestamp="2026-05-25 07:12:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 07:13:12.461657317 +0000 UTC m=+1630.697843713" watchObservedRunningTime="2026-05-25 07:13:12.464752776 +0000 UTC m=+1630.700939169"
May 25 07:13:14.956319 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:14.956235    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:13:14.956319 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:14.956284    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:13:14.956319 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:14.956298    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:13:14.958735 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:14.958690    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.39:8443: connect: connection refused"
May 25 07:13:14.958904 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:14.958736    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="online" probeResult="failure" output="Get \"https://10.134.0.39:6567/health\": dial tcp 10.134.0.39:6567: connect: connection refused"
May 25 07:13:14.958904 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:14.958703    2572 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.39:8816: connect: connection refused"
May 25 07:13:17.956928 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:17.956894    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:13:17.957319 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:17.957014    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:13:17.957470 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:17.957444    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:13:17.957538 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:17.957483    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:13:17.957859 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:17.957826    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:13:17.957965 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:17.957884    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:13:17.961066 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:17.961046    2572 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:13:18.463517 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:18.463481    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:13:18.466416 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:13:18.466395    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:14:20.889460 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:20.889420    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"]
May 25 07:14:20.890066 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:20.889878    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="online" containerID="cri-o://8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755" gracePeriod=30
May 25 07:14:20.890066 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:20.889908    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="offline" containerID="cri-o://3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185" gracePeriod=30
May 25 07:14:20.890066 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:20.889909    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="ui" containerID="cri-o://209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441" gracePeriod=30
May 25 07:14:21.047218 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:21.047179    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"]
May 25 07:14:21.047578 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:21.047524    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="registry" containerID="cri-o://9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c" gracePeriod=30
May 25 07:14:21.047669 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:21.047601    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="ui" containerID="cri-o://312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9" gracePeriod=30
May 25 07:14:21.047735 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:21.047609    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="online" containerID="cri-o://3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84" gracePeriod=30
May 25 07:14:21.047910 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:21.047829    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="offline" containerID="cri-o://992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1" gracePeriod=30
May 25 07:14:21.651828 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:21.651790    2572 generic.go:358] "Generic (PLEG): container finished" podID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerID="209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441" exitCode=0
May 25 07:14:21.651998 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:21.651863    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" event={"ID":"8b45503d-f54c-4359-a750-ab1f3f82c626","Type":"ContainerDied","Data":"209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441"}
May 25 07:14:22.656582 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:22.656551    2572 generic.go:358] "Generic (PLEG): container finished" podID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerID="8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755" exitCode=0
May 25 07:14:22.657042 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:22.656619    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" event={"ID":"8b45503d-f54c-4359-a750-ab1f3f82c626","Type":"ContainerDied","Data":"8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755"}
May 25 07:14:22.658908 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:22.658887    2572 generic.go:358] "Generic (PLEG): container finished" podID="9bbb034c-1823-4ffe-851b-1b2723236830" containerID="312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9" exitCode=0
May 25 07:14:22.658908 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:22.658907    2572 generic.go:358] "Generic (PLEG): container finished" podID="9bbb034c-1823-4ffe-851b-1b2723236830" containerID="3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84" exitCode=0
May 25 07:14:22.659070 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:22.658964    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerDied","Data":"312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9"}
May 25 07:14:22.659070 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:22.658986    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerDied","Data":"3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84"}
May 25 07:14:26.217598 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:26.217552    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 25 07:14:27.217124 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:27.217080    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 25 07:14:27.957974 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:27.957927    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.39:8443: connect: connection refused"
May 25 07:14:28.464398 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:28.464355    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="online" probeResult="failure" output="Get \"https://10.134.0.39:6567/health\": dial tcp 10.134.0.39:6567: connect: connection refused"
May 25 07:14:36.217699 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:36.217644    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 25 07:14:37.217202 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:37.217157    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 25 07:14:37.958047 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:37.957995    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.39:8443: connect: connection refused"
May 25 07:14:38.463864 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:38.463801    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="online" probeResult="failure" output="Get \"https://10.134.0.39:6567/health\": dial tcp 10.134.0.39:6567: connect: connection refused"
May 25 07:14:46.217384 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:46.217337    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 25 07:14:46.217879 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:46.217478    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:14:47.217617 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:47.217569    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 25 07:14:47.218096 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:47.217706    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:14:47.957745 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:47.957704    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.39:8443: connect: connection refused"
May 25 07:14:47.957971 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:47.957857    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:14:48.464336 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:48.464292    2572 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="online" probeResult="failure" output="Get \"https://10.134.0.39:6567/health\": dial tcp 10.134.0.39:6567: connect: connection refused"
May 25 07:14:48.464719 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:48.464430    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:14:51.532422 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.532400    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:14:51.671978 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.671888    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/8b45503d-f54c-4359-a750-ab1f3f82c626-registry-tls\") pod \"8b45503d-f54c-4359-a750-ab1f3f82c626\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") "
May 25 07:14:51.671978 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.671933    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-sxgjh\" (UniqueName: \"kubernetes.io/projected/8b45503d-f54c-4359-a750-ab1f3f82c626-kube-api-access-sxgjh\") pod \"8b45503d-f54c-4359-a750-ab1f3f82c626\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") "
May 25 07:14:51.671978 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.671963    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-online-tls\") pod \"8b45503d-f54c-4359-a750-ab1f3f82c626\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") "
May 25 07:14:51.672313 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.671994    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-ui-tls\") pod \"8b45503d-f54c-4359-a750-ab1f3f82c626\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") "
May 25 07:14:51.672313 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.672017    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-offline-tls\") pod \"8b45503d-f54c-4359-a750-ab1f3f82c626\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") "
May 25 07:14:51.672313 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.672042    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8b45503d-f54c-4359-a750-ab1f3f82c626-feast-data\") pod \"8b45503d-f54c-4359-a750-ab1f3f82c626\" (UID: \"8b45503d-f54c-4359-a750-ab1f3f82c626\") "
May 25 07:14:51.672313 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.672283    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/8b45503d-f54c-4359-a750-ab1f3f82c626-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8b45503d-f54c-4359-a750-ab1f3f82c626" (UID: "8b45503d-f54c-4359-a750-ab1f3f82c626"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 25 07:14:51.672905 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.672877    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8b45503d-f54c-4359-a750-ab1f3f82c626-feast-data" (OuterVolumeSpecName: "feast-data") pod "8b45503d-f54c-4359-a750-ab1f3f82c626" (UID: "8b45503d-f54c-4359-a750-ab1f3f82c626"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 07:14:51.674460 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.674432    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "8b45503d-f54c-4359-a750-ab1f3f82c626" (UID: "8b45503d-f54c-4359-a750-ab1f3f82c626"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:14:51.674573 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.674466    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "8b45503d-f54c-4359-a750-ab1f3f82c626" (UID: "8b45503d-f54c-4359-a750-ab1f3f82c626"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:14:51.674573 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.674505    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8b45503d-f54c-4359-a750-ab1f3f82c626-kube-api-access-sxgjh" (OuterVolumeSpecName: "kube-api-access-sxgjh") pod "8b45503d-f54c-4359-a750-ab1f3f82c626" (UID: "8b45503d-f54c-4359-a750-ab1f3f82c626"). InnerVolumeSpecName "kube-api-access-sxgjh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 07:14:51.674673 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.674655    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-online-tls" (OuterVolumeSpecName: "online-tls") pod "8b45503d-f54c-4359-a750-ab1f3f82c626" (UID: "8b45503d-f54c-4359-a750-ab1f3f82c626"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:14:51.687322 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.687304    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:14:51.743869 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.743816    2572 generic.go:358] "Generic (PLEG): container finished" podID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerID="3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185" exitCode=137
May 25 07:14:51.744043 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.743881    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" event={"ID":"8b45503d-f54c-4359-a750-ab1f3f82c626","Type":"ContainerDied","Data":"3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185"}
May 25 07:14:51.744043 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.743908    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"
May 25 07:14:51.744043 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.743920    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9" event={"ID":"8b45503d-f54c-4359-a750-ab1f3f82c626","Type":"ContainerDied","Data":"76545ef9b99e902c5c5c9a66888302c02b2a6ba0172ef98d781dcc3df76931c2"}
May 25 07:14:51.744043 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.743946    2572 scope.go:117] "RemoveContainer" containerID="209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441"
May 25 07:14:51.746610 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.746584    2572 generic.go:358] "Generic (PLEG): container finished" podID="9bbb034c-1823-4ffe-851b-1b2723236830" containerID="992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1" exitCode=137
May 25 07:14:51.746610 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.746608    2572 generic.go:358] "Generic (PLEG): container finished" podID="9bbb034c-1823-4ffe-851b-1b2723236830" containerID="9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c" exitCode=137
May 25 07:14:51.746765 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.746607    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerDied","Data":"992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1"}
May 25 07:14:51.746765 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.746643    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerDied","Data":"9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c"}
May 25 07:14:51.746765 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.746659    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm" event={"ID":"9bbb034c-1823-4ffe-851b-1b2723236830","Type":"ContainerDied","Data":"af7dc785724d56022376887168d7092162b9dd7ee9ed3d957f25781fab613d66"}
May 25 07:14:51.746765 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.746692    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"
May 25 07:14:51.753006 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.752989    2572 scope.go:117] "RemoveContainer" containerID="3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185"
May 25 07:14:51.760175 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.760150    2572 scope.go:117] "RemoveContainer" containerID="8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755"
May 25 07:14:51.764147 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.764129    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"]
May 25 07:14:51.767951 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.767910    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-8l9d9"]
May 25 07:14:51.767951 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.767934    2572 scope.go:117] "RemoveContainer" containerID="2a1af6028c85b3f4f553cc5a458af0f9f3d9f1f2eeb5fa7223125e37ba6f7a4d"
May 25 07:14:51.773035 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.773008    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-online-tls\") pod \"9bbb034c-1823-4ffe-851b-1b2723236830\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") "
May 25 07:14:51.773122 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.773043    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p8hvw\" (UniqueName: \"kubernetes.io/projected/9bbb034c-1823-4ffe-851b-1b2723236830-kube-api-access-p8hvw\") pod \"9bbb034c-1823-4ffe-851b-1b2723236830\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") "
May 25 07:14:51.773168 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.773123    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-ui-tls\") pod \"9bbb034c-1823-4ffe-851b-1b2723236830\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") "
May 25 07:14:51.773215 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.773173    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-registry-tls\") pod \"9bbb034c-1823-4ffe-851b-1b2723236830\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") "
May 25 07:14:51.773215 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.773205    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9bbb034c-1823-4ffe-851b-1b2723236830-feast-data\") pod \"9bbb034c-1823-4ffe-851b-1b2723236830\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") "
May 25 07:14:51.773316 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.773229    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-offline-tls\") pod \"9bbb034c-1823-4ffe-851b-1b2723236830\" (UID: \"9bbb034c-1823-4ffe-851b-1b2723236830\") "
May 25 07:14:51.773479 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.773461    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/8b45503d-f54c-4359-a750-ab1f3f82c626-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:14:51.773594 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.773486    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-sxgjh\" (UniqueName: \"kubernetes.io/projected/8b45503d-f54c-4359-a750-ab1f3f82c626-kube-api-access-sxgjh\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:14:51.773594 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.773501    2572 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-online-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:14:51.773594 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.773516    2572 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-ui-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:14:51.773594 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.773529    2572 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8b45503d-f54c-4359-a750-ab1f3f82c626-offline-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:14:51.773594 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.773543    2572 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8b45503d-f54c-4359-a750-ab1f3f82c626-feast-data\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:14:51.774126 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.774099    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9bbb034c-1823-4ffe-851b-1b2723236830-feast-data" (OuterVolumeSpecName: "feast-data") pod "9bbb034c-1823-4ffe-851b-1b2723236830" (UID: "9bbb034c-1823-4ffe-851b-1b2723236830"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 07:14:51.775527 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.775489    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "9bbb034c-1823-4ffe-851b-1b2723236830" (UID: "9bbb034c-1823-4ffe-851b-1b2723236830"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:14:51.775666 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.775564    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "9bbb034c-1823-4ffe-851b-1b2723236830" (UID: "9bbb034c-1823-4ffe-851b-1b2723236830"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:14:51.775666 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.775613    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-online-tls" (OuterVolumeSpecName: "online-tls") pod "9bbb034c-1823-4ffe-851b-1b2723236830" (UID: "9bbb034c-1823-4ffe-851b-1b2723236830"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:14:51.776093 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.775827    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9bbb034c-1823-4ffe-851b-1b2723236830" (UID: "9bbb034c-1823-4ffe-851b-1b2723236830"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 25 07:14:51.776093 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.776062    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9bbb034c-1823-4ffe-851b-1b2723236830-kube-api-access-p8hvw" (OuterVolumeSpecName: "kube-api-access-p8hvw") pod "9bbb034c-1823-4ffe-851b-1b2723236830" (UID: "9bbb034c-1823-4ffe-851b-1b2723236830"). InnerVolumeSpecName "kube-api-access-p8hvw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 07:14:51.776515 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.776500    2572 scope.go:117] "RemoveContainer" containerID="d1d53c4d18993efa032f8f50a892698d6762ade878c28bc161bf93c782469a31"
May 25 07:14:51.783820 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.783799    2572 scope.go:117] "RemoveContainer" containerID="209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441"
May 25 07:14:51.784111 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:14:51.784092    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441\": container with ID starting with 209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441 not found: ID does not exist" containerID="209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441"
May 25 07:14:51.784189 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.784123    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441"} err="failed to get container status \"209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441\": rpc error: code = NotFound desc = could not find container \"209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441\": container with ID starting with 209c98a2ee8ea3869c78c84fd1fa61d091d9b44010467384e750e8c92045f441 not found: ID does not exist"
May 25 07:14:51.784189 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.784149    2572 scope.go:117] "RemoveContainer" containerID="3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185"
May 25 07:14:51.784409 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:14:51.784391    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185\": container with ID starting with 3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185 not found: ID does not exist" containerID="3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185"
May 25 07:14:51.784515 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.784414    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185"} err="failed to get container status \"3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185\": rpc error: code = NotFound desc = could not find container \"3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185\": container with ID starting with 3b717a2599dfd8700f893eae8ee880a7832806ee3408046de6c19c24a1a4c185 not found: ID does not exist"
May 25 07:14:51.784515 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.784434    2572 scope.go:117] "RemoveContainer" containerID="8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755"
May 25 07:14:51.784666 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:14:51.784647    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755\": container with ID starting with 8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755 not found: ID does not exist" containerID="8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755"
May 25 07:14:51.784727 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.784671    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755"} err="failed to get container status \"8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755\": rpc error: code = NotFound desc = could not find container \"8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755\": container with ID starting with 8188538d8ae585323aa1e0a204b598ce7de556d753612fe467e6536f163b8755 not found: ID does not exist"
May 25 07:14:51.784727 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.784692    2572 scope.go:117] "RemoveContainer" containerID="2a1af6028c85b3f4f553cc5a458af0f9f3d9f1f2eeb5fa7223125e37ba6f7a4d"
May 25 07:14:51.784996 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:14:51.784969    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2a1af6028c85b3f4f553cc5a458af0f9f3d9f1f2eeb5fa7223125e37ba6f7a4d\": container with ID starting with 2a1af6028c85b3f4f553cc5a458af0f9f3d9f1f2eeb5fa7223125e37ba6f7a4d not found: ID does not exist" containerID="2a1af6028c85b3f4f553cc5a458af0f9f3d9f1f2eeb5fa7223125e37ba6f7a4d"
May 25 07:14:51.785107 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.784999    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2a1af6028c85b3f4f553cc5a458af0f9f3d9f1f2eeb5fa7223125e37ba6f7a4d"} err="failed to get container status \"2a1af6028c85b3f4f553cc5a458af0f9f3d9f1f2eeb5fa7223125e37ba6f7a4d\": rpc error: code = NotFound desc = could not find container \"2a1af6028c85b3f4f553cc5a458af0f9f3d9f1f2eeb5fa7223125e37ba6f7a4d\": container with ID starting with 2a1af6028c85b3f4f553cc5a458af0f9f3d9f1f2eeb5fa7223125e37ba6f7a4d not found: ID does not exist"
May 25 07:14:51.785107 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.785015    2572 scope.go:117] "RemoveContainer" containerID="d1d53c4d18993efa032f8f50a892698d6762ade878c28bc161bf93c782469a31"
May 25 07:14:51.785252 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:14:51.785234    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d1d53c4d18993efa032f8f50a892698d6762ade878c28bc161bf93c782469a31\": container with ID starting with d1d53c4d18993efa032f8f50a892698d6762ade878c28bc161bf93c782469a31 not found: ID does not exist" containerID="d1d53c4d18993efa032f8f50a892698d6762ade878c28bc161bf93c782469a31"
May 25 07:14:51.785316 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.785255    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d1d53c4d18993efa032f8f50a892698d6762ade878c28bc161bf93c782469a31"} err="failed to get container status \"d1d53c4d18993efa032f8f50a892698d6762ade878c28bc161bf93c782469a31\": rpc error: code = NotFound desc = could not find container \"d1d53c4d18993efa032f8f50a892698d6762ade878c28bc161bf93c782469a31\": container with ID starting with d1d53c4d18993efa032f8f50a892698d6762ade878c28bc161bf93c782469a31 not found: ID does not exist"
May 25 07:14:51.785316 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.785271    2572 scope.go:117] "RemoveContainer" containerID="312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9"
May 25 07:14:51.792692 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.792664    2572 scope.go:117] "RemoveContainer" containerID="992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1"
May 25 07:14:51.799607 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.799590    2572 scope.go:117] "RemoveContainer" containerID="3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84"
May 25 07:14:51.806120 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.806103    2572 scope.go:117] "RemoveContainer" containerID="9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c"
May 25 07:14:51.814505 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.814475    2572 scope.go:117] "RemoveContainer" containerID="03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9"
May 25 07:14:51.821379 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.821358    2572 scope.go:117] "RemoveContainer" containerID="0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2"
May 25 07:14:51.828059 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.828041    2572 scope.go:117] "RemoveContainer" containerID="312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9"
May 25 07:14:51.828291 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:14:51.828272    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9\": container with ID starting with 312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9 not found: ID does not exist" containerID="312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9"
May 25 07:14:51.828336 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.828301    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9"} err="failed to get container status \"312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9\": rpc error: code = NotFound desc = could not find container \"312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9\": container with ID starting with 312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9 not found: ID does not exist"
May 25 07:14:51.828336 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.828320    2572 scope.go:117] "RemoveContainer" containerID="992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1"
May 25 07:14:51.828518 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:14:51.828500    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1\": container with ID starting with 992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1 not found: ID does not exist" containerID="992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1"
May 25 07:14:51.828560 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.828532    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1"} err="failed to get container status \"992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1\": rpc error: code = NotFound desc = could not find container \"992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1\": container with ID starting with 992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1 not found: ID does not exist"
May 25 07:14:51.828560 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.828546    2572 scope.go:117] "RemoveContainer" containerID="3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84"
May 25 07:14:51.828772 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:14:51.828753    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84\": container with ID starting with 3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84 not found: ID does not exist" containerID="3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84"
May 25 07:14:51.828810 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.828776    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84"} err="failed to get container status \"3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84\": rpc error: code = NotFound desc = could not find container \"3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84\": container with ID starting with 3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84 not found: ID does not exist"
May 25 07:14:51.828810 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.828789    2572 scope.go:117] "RemoveContainer" containerID="9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c"
May 25 07:14:51.829135 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:14:51.829119    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c\": container with ID starting with 9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c not found: ID does not exist" containerID="9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c"
May 25 07:14:51.829210 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.829136    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c"} err="failed to get container status \"9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c\": rpc error: code = NotFound desc = could not find container \"9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c\": container with ID starting with 9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c not found: ID does not exist"
May 25 07:14:51.829210 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.829149    2572 scope.go:117] "RemoveContainer" containerID="03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9"
May 25 07:14:51.829374 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:14:51.829358    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9\": container with ID starting with 03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9 not found: ID does not exist" containerID="03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9"
May 25 07:14:51.829414 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.829379    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9"} err="failed to get container status \"03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9\": rpc error: code = NotFound desc = could not find container \"03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9\": container with ID starting with 03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9 not found: ID does not exist"
May 25 07:14:51.829414 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.829395    2572 scope.go:117] "RemoveContainer" containerID="0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2"
May 25 07:14:51.829591 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:14:51.829572    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2\": container with ID starting with 0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2 not found: ID does not exist" containerID="0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2"
May 25 07:14:51.829699 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.829595    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2"} err="failed to get container status \"0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2\": rpc error: code = NotFound desc = could not find container \"0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2\": container with ID starting with 0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2 not found: ID does not exist"
May 25 07:14:51.829699 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.829614    2572 scope.go:117] "RemoveContainer" containerID="312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9"
May 25 07:14:51.829816 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.829800    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9"} err="failed to get container status \"312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9\": rpc error: code = NotFound desc = could not find container \"312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9\": container with ID starting with 312c796c05dd50a9a65afef0cfc8eaff9211328728348ba27044219c4d126bf9 not found: ID does not exist"
May 25 07:14:51.829926 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.829816    2572 scope.go:117] "RemoveContainer" containerID="992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1"
May 25 07:14:51.830057 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.830039    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1"} err="failed to get container status \"992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1\": rpc error: code = NotFound desc = could not find container \"992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1\": container with ID starting with 992d4f8445d2ad49369f82ee48cc489690afad6d74669f9afcd29c0a14acd7e1 not found: ID does not exist"
May 25 07:14:51.830124 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.830059    2572 scope.go:117] "RemoveContainer" containerID="3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84"
May 25 07:14:51.830293 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.830276    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84"} err="failed to get container status \"3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84\": rpc error: code = NotFound desc = could not find container \"3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84\": container with ID starting with 3a0fbfef43ad30b33052fbee852fc1b522c01a41f4a81308f7bdc62db3a05f84 not found: ID does not exist"
May 25 07:14:51.830335 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.830294    2572 scope.go:117] "RemoveContainer" containerID="9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c"
May 25 07:14:51.830474 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.830458    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c"} err="failed to get container status \"9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c\": rpc error: code = NotFound desc = could not find container \"9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c\": container with ID starting with 9391c1979271abb4e2c85ac0ae2d65912935a96c8951d1d32ff6a44de7c5045c not found: ID does not exist"
May 25 07:14:51.830555 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.830475    2572 scope.go:117] "RemoveContainer" containerID="03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9"
May 25 07:14:51.830666 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.830648    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9"} err="failed to get container status \"03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9\": rpc error: code = NotFound desc = could not find container \"03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9\": container with ID starting with 03a94c79acc2b5b2561780aae4f71d7b729af6ac76fe96c9c8a0c9061c7534e9 not found: ID does not exist"
May 25 07:14:51.830730 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.830667    2572 scope.go:117] "RemoveContainer" containerID="0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2"
May 25 07:14:51.830890 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.830870    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2"} err="failed to get container status \"0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2\": rpc error: code = NotFound desc = could not find container \"0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2\": container with ID starting with 0762d715299cd5cab08338a743542a801f4724406c795bb403b2d74ce328aaf2 not found: ID does not exist"
May 25 07:14:51.874938 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.874902    2572 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-registry-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:14:51.874938 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.874932    2572 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9bbb034c-1823-4ffe-851b-1b2723236830-feast-data\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:14:51.874938 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.874943    2572 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-offline-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:14:51.875167 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.874951    2572 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-online-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:14:51.875167 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.874962    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p8hvw\" (UniqueName: \"kubernetes.io/projected/9bbb034c-1823-4ffe-851b-1b2723236830-kube-api-access-p8hvw\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:14:51.875167 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:51.874972    2572 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9bbb034c-1823-4ffe-851b-1b2723236830-ui-tls\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:14:52.067264 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:52.067226    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"]
May 25 07:14:52.070493 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:52.070460    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-slsrm"]
May 25 07:14:52.282318 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:52.282283    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" path="/var/lib/kubelet/pods/8b45503d-f54c-4359-a750-ab1f3f82c626/volumes"
May 25 07:14:52.282829 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:14:52.282815    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" path="/var/lib/kubelet/pods/9bbb034c-1823-4ffe-851b-1b2723236830/volumes"
May 25 07:15:10.419599 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419558    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-tstrs/must-gather-88hbb"]
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419829    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="online"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419852    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="online"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419863    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="online"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419868    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="online"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419877    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="feast-apply"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419884    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="feast-apply"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419892    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="feast-init"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419898    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="feast-init"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419905    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="feast-init"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419910    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="feast-init"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419916    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="offline"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419921    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="offline"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419929    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="registry"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419934    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="registry"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419941    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="ui"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419946    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="ui"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419951    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="feast-apply"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419956    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="feast-apply"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419962    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="ui"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419967    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="ui"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419975    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="offline"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.419980    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="offline"
May 25 07:15:10.420013 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.420022    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="online"
May 25 07:15:10.420700 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.420031    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="ui"
May 25 07:15:10.420700 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.420038    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="offline"
May 25 07:15:10.420700 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.420044    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="registry"
May 25 07:15:10.420700 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.420051    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="online"
May 25 07:15:10.420700 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.420057    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="8b45503d-f54c-4359-a750-ab1f3f82c626" containerName="offline"
May 25 07:15:10.420700 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.420064    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="9bbb034c-1823-4ffe-851b-1b2723236830" containerName="ui"
May 25 07:15:10.424093 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.424077    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tstrs/must-gather-88hbb"
May 25 07:15:10.425783 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.425758    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tstrs\"/\"kube-root-ca.crt\""
May 25 07:15:10.426225 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.426209    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-tstrs\"/\"openshift-service-ca.crt\""
May 25 07:15:10.426295 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.426212    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-tstrs\"/\"default-dockercfg-98l2p\""
May 25 07:15:10.430554 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.430526    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tstrs/must-gather-88hbb"]
May 25 07:15:10.523503 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.523468    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dz4jb\" (UniqueName: \"kubernetes.io/projected/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6-kube-api-access-dz4jb\") pod \"must-gather-88hbb\" (UID: \"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6\") " pod="openshift-must-gather-tstrs/must-gather-88hbb"
May 25 07:15:10.523703 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.523520    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6-must-gather-output\") pod \"must-gather-88hbb\" (UID: \"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6\") " pod="openshift-must-gather-tstrs/must-gather-88hbb"
May 25 07:15:10.624569 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.624531    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dz4jb\" (UniqueName: \"kubernetes.io/projected/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6-kube-api-access-dz4jb\") pod \"must-gather-88hbb\" (UID: \"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6\") " pod="openshift-must-gather-tstrs/must-gather-88hbb"
May 25 07:15:10.624697 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.624576    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6-must-gather-output\") pod \"must-gather-88hbb\" (UID: \"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6\") " pod="openshift-must-gather-tstrs/must-gather-88hbb"
May 25 07:15:10.625023 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.625004    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6-must-gather-output\") pod \"must-gather-88hbb\" (UID: \"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6\") " pod="openshift-must-gather-tstrs/must-gather-88hbb"
May 25 07:15:10.632974 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.632951    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dz4jb\" (UniqueName: \"kubernetes.io/projected/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6-kube-api-access-dz4jb\") pod \"must-gather-88hbb\" (UID: \"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6\") " pod="openshift-must-gather-tstrs/must-gather-88hbb"
May 25 07:15:10.748307 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.748204    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tstrs/must-gather-88hbb"
May 25 07:15:10.872436 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:10.872403    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-tstrs/must-gather-88hbb"]
May 25 07:15:10.876273 ip-10-0-128-6 kubenswrapper[2572]: W0525 07:15:10.876248    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podaa9cd6f9_a0dc_4f12_b854_a6af75e200c6.slice/crio-2b391a8e30e2f29aae5525f1ce5c70a2ffc1bdbe2f7ba989c1fc33b6349a1ca9 WatchSource:0}: Error finding container 2b391a8e30e2f29aae5525f1ce5c70a2ffc1bdbe2f7ba989c1fc33b6349a1ca9: Status 404 returned error can't find the container with id 2b391a8e30e2f29aae5525f1ce5c70a2ffc1bdbe2f7ba989c1fc33b6349a1ca9
May 25 07:15:11.806675 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:11.806636    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tstrs/must-gather-88hbb" event={"ID":"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6","Type":"ContainerStarted","Data":"2b391a8e30e2f29aae5525f1ce5c70a2ffc1bdbe2f7ba989c1fc33b6349a1ca9"}
May 25 07:15:17.115378 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:17.115346    2572 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 25 07:15:17.830555 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:17.830514    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tstrs/must-gather-88hbb" event={"ID":"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6","Type":"ContainerStarted","Data":"2e6d2ade8eb6350e40957d9bc2524356579cf35e1dcda0fe382d1e66fb029d1b"}
May 25 07:15:17.830740 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:17.830563    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tstrs/must-gather-88hbb" event={"ID":"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6","Type":"ContainerStarted","Data":"c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65"}
May 25 07:15:17.845043 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:17.844989    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-tstrs/must-gather-88hbb" podStartSLOduration=1.729145136 podStartE2EDuration="7.844976093s" podCreationTimestamp="2026-05-25 07:15:10 +0000 UTC" firstStartedPulling="2026-05-25 07:15:10.877945602 +0000 UTC m=+1749.114131987" lastFinishedPulling="2026-05-25 07:15:16.993776574 +0000 UTC m=+1755.229962944" observedRunningTime="2026-05-25 07:15:17.842954471 +0000 UTC m=+1756.079140863" watchObservedRunningTime="2026-05-25 07:15:17.844976093 +0000 UTC m=+1756.081162484"
May 25 07:15:26.858196 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:26.858161    2572 generic.go:358] "Generic (PLEG): container finished" podID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" containerID="c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65" exitCode=0
May 25 07:15:26.858644 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:26.858233    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-tstrs/must-gather-88hbb" event={"ID":"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6","Type":"ContainerDied","Data":"c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65"}
May 25 07:15:26.858644 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:26.858576    2572 scope.go:117] "RemoveContainer" containerID="c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65"
May 25 07:15:27.683649 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:27.683616    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-tstrs_must-gather-88hbb_aa9cd6f9-a0dc-4f12-b854-a6af75e200c6/gather/0.log"
May 25 07:15:30.533269 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:30.533239    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-6rmxj_0406bbb3-eaa6-43dc-aef7-9b2b2b1a8418/global-pull-secret-syncer/0.log"
May 25 07:15:30.690210 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:30.690177    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-77nvd_a7c58bab-87c9-44c8-90d8-5983437cd492/konnectivity-agent/0.log"
May 25 07:15:30.730382 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:30.730353    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-128-6.ec2.internal_732aed7a0244464bbcdfac9017fa7a54/haproxy/0.log"
May 25 07:15:32.989485 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:32.989445    2572 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-tstrs/must-gather-88hbb"]
May 25 07:15:32.990002 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:32.989755    2572 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-must-gather-tstrs/must-gather-88hbb" podUID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" containerName="copy" containerID="cri-o://2e6d2ade8eb6350e40957d9bc2524356579cf35e1dcda0fe382d1e66fb029d1b" gracePeriod=2
May 25 07:15:32.991276 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:32.991245    2572 status_manager.go:895] "Failed to get status for pod" podUID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" pod="openshift-must-gather-tstrs/must-gather-88hbb" err="pods \"must-gather-88hbb\" is forbidden: User \"system:node:ip-10-0-128-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-tstrs\": no relationship found between node 'ip-10-0-128-6.ec2.internal' and this object"
May 25 07:15:32.991965 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:32.991943    2572 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-tstrs/must-gather-88hbb"]
May 25 07:15:33.217367 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.217345    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-tstrs_must-gather-88hbb_aa9cd6f9-a0dc-4f12-b854-a6af75e200c6/copy/0.log"
May 25 07:15:33.217698 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.217683    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tstrs/must-gather-88hbb"
May 25 07:15:33.219158 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.219133    2572 status_manager.go:895] "Failed to get status for pod" podUID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" pod="openshift-must-gather-tstrs/must-gather-88hbb" err="pods \"must-gather-88hbb\" is forbidden: User \"system:node:ip-10-0-128-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-tstrs\": no relationship found between node 'ip-10-0-128-6.ec2.internal' and this object"
May 25 07:15:33.309153 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.309077    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6-must-gather-output\") pod \"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6\" (UID: \"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6\") "
May 25 07:15:33.309153 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.309137    2572 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dz4jb\" (UniqueName: \"kubernetes.io/projected/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6-kube-api-access-dz4jb\") pod \"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6\" (UID: \"aa9cd6f9-a0dc-4f12-b854-a6af75e200c6\") "
May 25 07:15:33.309461 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.309438    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" (UID: "aa9cd6f9-a0dc-4f12-b854-a6af75e200c6"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 25 07:15:33.311422 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.311402    2572 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6-kube-api-access-dz4jb" (OuterVolumeSpecName: "kube-api-access-dz4jb") pod "aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" (UID: "aa9cd6f9-a0dc-4f12-b854-a6af75e200c6"). InnerVolumeSpecName "kube-api-access-dz4jb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 25 07:15:33.409788 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.409752    2572 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6-must-gather-output\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:15:33.409788 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.409784    2572 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dz4jb\" (UniqueName: \"kubernetes.io/projected/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6-kube-api-access-dz4jb\") on node \"ip-10-0-128-6.ec2.internal\" DevicePath \"\""
May 25 07:15:33.714024 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.713987    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_cluster-monitoring-operator-6c75959455-4cncm_7cde0c58-8416-438c-ba27-4e5de27d3d91/cluster-monitoring-operator/0.log"
May 25 07:15:33.801466 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.801440    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-5858b6cf8-nl6rr_40477ac9-6d9e-438c-a2ad-3f18c1dcb0a0/metrics-server/0.log"
May 25 07:15:33.826364 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.826329    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-fs5x7_b7be6e3f-d7bd-4709-bc19-aef212c81644/monitoring-plugin/0.log"
May 25 07:15:33.854581 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.854556    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cwf8g_97787552-67e5-4f37-a9e9-8f53b7d66ca1/node-exporter/0.log"
May 25 07:15:33.874529 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.874503    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cwf8g_97787552-67e5-4f37-a9e9-8f53b7d66ca1/kube-rbac-proxy/0.log"
May 25 07:15:33.876811 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.876793    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-tstrs_must-gather-88hbb_aa9cd6f9-a0dc-4f12-b854-a6af75e200c6/copy/0.log"
May 25 07:15:33.877102 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.877082    2572 generic.go:358] "Generic (PLEG): container finished" podID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" containerID="2e6d2ade8eb6350e40957d9bc2524356579cf35e1dcda0fe382d1e66fb029d1b" exitCode=143
May 25 07:15:33.877170 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.877147    2572 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-tstrs/must-gather-88hbb"
May 25 07:15:33.877235 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.877172    2572 scope.go:117] "RemoveContainer" containerID="2e6d2ade8eb6350e40957d9bc2524356579cf35e1dcda0fe382d1e66fb029d1b"
May 25 07:15:33.878790 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.878766    2572 status_manager.go:895] "Failed to get status for pod" podUID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" pod="openshift-must-gather-tstrs/must-gather-88hbb" err="pods \"must-gather-88hbb\" is forbidden: User \"system:node:ip-10-0-128-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-tstrs\": no relationship found between node 'ip-10-0-128-6.ec2.internal' and this object"
May 25 07:15:33.885647 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.885459    2572 scope.go:117] "RemoveContainer" containerID="c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65"
May 25 07:15:33.887695 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.887669    2572 status_manager.go:895] "Failed to get status for pod" podUID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" pod="openshift-must-gather-tstrs/must-gather-88hbb" err="pods \"must-gather-88hbb\" is forbidden: User \"system:node:ip-10-0-128-6.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-tstrs\": no relationship found between node 'ip-10-0-128-6.ec2.internal' and this object"
May 25 07:15:33.893960 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.893943    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-cwf8g_97787552-67e5-4f37-a9e9-8f53b7d66ca1/init-textfile/0.log"
May 25 07:15:33.896598 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.896583    2572 scope.go:117] "RemoveContainer" containerID="2e6d2ade8eb6350e40957d9bc2524356579cf35e1dcda0fe382d1e66fb029d1b"
May 25 07:15:33.896860 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:15:33.896828    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"2e6d2ade8eb6350e40957d9bc2524356579cf35e1dcda0fe382d1e66fb029d1b\": container with ID starting with 2e6d2ade8eb6350e40957d9bc2524356579cf35e1dcda0fe382d1e66fb029d1b not found: ID does not exist" containerID="2e6d2ade8eb6350e40957d9bc2524356579cf35e1dcda0fe382d1e66fb029d1b"
May 25 07:15:33.896924 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.896867    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"2e6d2ade8eb6350e40957d9bc2524356579cf35e1dcda0fe382d1e66fb029d1b"} err="failed to get container status \"2e6d2ade8eb6350e40957d9bc2524356579cf35e1dcda0fe382d1e66fb029d1b\": rpc error: code = NotFound desc = could not find container \"2e6d2ade8eb6350e40957d9bc2524356579cf35e1dcda0fe382d1e66fb029d1b\": container with ID starting with 2e6d2ade8eb6350e40957d9bc2524356579cf35e1dcda0fe382d1e66fb029d1b not found: ID does not exist"
May 25 07:15:33.896924 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.896885    2572 scope.go:117] "RemoveContainer" containerID="c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65"
May 25 07:15:33.897099 ip-10-0-128-6 kubenswrapper[2572]: E0525 07:15:33.897084    2572 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65\": container with ID starting with c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65 not found: ID does not exist" containerID="c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65"
May 25 07:15:33.897178 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:33.897104    2572 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65"} err="failed to get container status \"c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65\": rpc error: code = NotFound desc = could not find container \"c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65\": container with ID starting with c1b595c4f03960576df6d16b6323c8c7a1d47915a31d5da0916093232043fe65 not found: ID does not exist"
May 25 07:15:34.059957 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:34.059873    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-nwb6h_369c0515-c0a1-4042-bcd9-8efbe2663bd8/kube-rbac-proxy-main/0.log"
May 25 07:15:34.093486 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:34.093455    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-nwb6h_369c0515-c0a1-4042-bcd9-8efbe2663bd8/kube-rbac-proxy-self/0.log"
May 25 07:15:34.115645 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:34.115622    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-nwb6h_369c0515-c0a1-4042-bcd9-8efbe2663bd8/openshift-state-metrics/0.log"
May 25 07:15:34.281803 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:34.281770    2572 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" path="/var/lib/kubelet/pods/aa9cd6f9-a0dc-4f12-b854-a6af75e200c6/volumes"
May 25 07:15:35.687578 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:35.687548    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-console_networking-console-plugin-d4d8f94f9-lg6d6_02cc22a1-cdc8-4c2b-8a2c-1a5b5a830bab/networking-console-plugin/0.log"
May 25 07:15:36.403828 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:36.403797    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-dcdb769b6-2sqn8_cb56ca3e-99a5-4239-850b-c065b8dc7353/console/0.log"
May 25 07:15:37.160350 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.160314    2572 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"]
May 25 07:15:37.160731 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.160585    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" containerName="gather"
May 25 07:15:37.160731 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.160597    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" containerName="gather"
May 25 07:15:37.160731 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.160617    2572 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" containerName="copy"
May 25 07:15:37.160731 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.160622    2572 state_mem.go:107] "Deleted CPUSet assignment" podUID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" containerName="copy"
May 25 07:15:37.160731 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.160664    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" containerName="copy"
May 25 07:15:37.160731 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.160673    2572 memory_manager.go:356] "RemoveStaleState removing state" podUID="aa9cd6f9-a0dc-4f12-b854-a6af75e200c6" containerName="gather"
May 25 07:15:37.163714 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.163692    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.165419 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.165393    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7f8fv\"/\"openshift-service-ca.crt\""
May 25 07:15:37.165909 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.165895    2572 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-7f8fv\"/\"default-dockercfg-2wv8v\""
May 25 07:15:37.165971 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.165934    2572 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-7f8fv\"/\"kube-root-ca.crt\""
May 25 07:15:37.171008 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.170984    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"]
May 25 07:15:37.237320 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.237276    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/767cd0e8-133d-4ead-8219-90be239b7b23-proc\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.237516 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.237378    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/767cd0e8-133d-4ead-8219-90be239b7b23-lib-modules\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.237516 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.237408    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/767cd0e8-133d-4ead-8219-90be239b7b23-podres\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.237516 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.237451    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xd9z7\" (UniqueName: \"kubernetes.io/projected/767cd0e8-133d-4ead-8219-90be239b7b23-kube-api-access-xd9z7\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.237516 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.237513    2572 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/767cd0e8-133d-4ead-8219-90be239b7b23-sys\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.327106 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.327081    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gz8v5_88e807ae-7d7e-426f-a3ab-25fa0d336a7f/dns/0.log"
May 25 07:15:37.338106 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.338083    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/767cd0e8-133d-4ead-8219-90be239b7b23-lib-modules\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.338190 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.338113    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/767cd0e8-133d-4ead-8219-90be239b7b23-podres\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.338190 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.338131    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xd9z7\" (UniqueName: \"kubernetes.io/projected/767cd0e8-133d-4ead-8219-90be239b7b23-kube-api-access-xd9z7\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.338190 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.338153    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/767cd0e8-133d-4ead-8219-90be239b7b23-sys\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.338345 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.338193    2572 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/767cd0e8-133d-4ead-8219-90be239b7b23-proc\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.338345 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.338220    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/767cd0e8-133d-4ead-8219-90be239b7b23-sys\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.338345 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.338257    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/767cd0e8-133d-4ead-8219-90be239b7b23-podres\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.338345 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.338257    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/767cd0e8-133d-4ead-8219-90be239b7b23-proc\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.338345 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.338283    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/767cd0e8-133d-4ead-8219-90be239b7b23-lib-modules\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.344608 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.344582    2572 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xd9z7\" (UniqueName: \"kubernetes.io/projected/767cd0e8-133d-4ead-8219-90be239b7b23-kube-api-access-xd9z7\") pod \"perf-node-gather-daemonset-h4tz4\" (UID: \"767cd0e8-133d-4ead-8219-90be239b7b23\") " pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.344896 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.344881    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-gz8v5_88e807ae-7d7e-426f-a3ab-25fa0d336a7f/kube-rbac-proxy/0.log"
May 25 07:15:37.399560 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.399533    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-5gkzs_a569a74b-5db9-428c-8a4e-9692a5636e5d/dns-node-resolver/0.log"
May 25 07:15:37.474308 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.474207    2572 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.597972 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.597945    2572 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"]
May 25 07:15:37.602022 ip-10-0-128-6 kubenswrapper[2572]: W0525 07:15:37.601984    2572 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pod767cd0e8_133d_4ead_8219_90be239b7b23.slice/crio-c78efa92ab870645b3ee74278660e7628b6cc839d31a8e5df99eaba61caccf40 WatchSource:0}: Error finding container c78efa92ab870645b3ee74278660e7628b6cc839d31a8e5df99eaba61caccf40: Status 404 returned error can't find the container with id c78efa92ab870645b3ee74278660e7628b6cc839d31a8e5df99eaba61caccf40
May 25 07:15:37.789352 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.789272    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-78494c4cc8-g2vcq_ebfad36b-e93a-41b2-8a88-94d1352d3f7d/registry/0.log"
May 25 07:15:37.823708 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.823681    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-87m8j_c3130f84-1146-48b3-97c0-bf676f8eb29d/node-ca/0.log"
May 25 07:15:37.890636 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.890598    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4" event={"ID":"767cd0e8-133d-4ead-8219-90be239b7b23","Type":"ContainerStarted","Data":"cd5c4fd7b9f1ef6de15e58779d4bc02b8e1d503bba643ea2c7065e59a878aaa5"}
May 25 07:15:37.890636 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.890633    2572 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4" event={"ID":"767cd0e8-133d-4ead-8219-90be239b7b23","Type":"ContainerStarted","Data":"c78efa92ab870645b3ee74278660e7628b6cc839d31a8e5df99eaba61caccf40"}
May 25 07:15:37.890931 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.890705    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:37.904086 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:37.904040    2572 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4" podStartSLOduration=0.904026399 podStartE2EDuration="904.026399ms" podCreationTimestamp="2026-05-25 07:15:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-25 07:15:37.902680702 +0000 UTC m=+1776.138867094" watchObservedRunningTime="2026-05-25 07:15:37.904026399 +0000 UTC m=+1776.140212790"
May 25 07:15:38.751472 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:38.751449    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-x4v6p_f1ba578b-9195-4d91-be43-38f7a5820121/serve-healthcheck-canary/0.log"
May 25 07:15:39.160369 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:39.160341    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jv7ph_aa459401-1adc-422c-b88d-1965967d9eba/kube-rbac-proxy/0.log"
May 25 07:15:39.178026 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:39.177999    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jv7ph_aa459401-1adc-422c-b88d-1965967d9eba/exporter/0.log"
May 25 07:15:39.196120 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:39.196092    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-jv7ph_aa459401-1adc-422c-b88d-1965967d9eba/extractor/0.log"
May 25 07:15:42.709511 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:42.709483    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-wk9ms_05ee7237-b28a-4df8-b33e-5adc887b50ad/migrator/0.log"
May 25 07:15:42.727112 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:42.727088    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator_migrator-b985f5d89-wk9ms_05ee7237-b28a-4df8-b33e-5adc887b50ad/graceful-termination/0.log"
May 25 07:15:42.978749 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:42.978674    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-7vk74_a8af8432-7b6d-4e9c-acb6-fedaa77d523f/kube-storage-version-migrator-operator/1.log"
May 25 07:15:42.979517 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:42.979497    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-7vk74_a8af8432-7b6d-4e9c-acb6-fedaa77d523f/kube-storage-version-migrator-operator/0.log"
May 25 07:15:43.714882 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:43.714854    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-4tfcl_c2acf7a4-48ea-4c5e-8d4c-d8232f383295/kube-multus/0.log"
May 25 07:15:43.755850 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:43.755823    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jwq4_a6aeda4f-547c-488a-9414-e6ab811425c6/kube-multus-additional-cni-plugins/0.log"
May 25 07:15:43.774253 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:43.774228    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jwq4_a6aeda4f-547c-488a-9414-e6ab811425c6/egress-router-binary-copy/0.log"
May 25 07:15:43.793330 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:43.793305    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jwq4_a6aeda4f-547c-488a-9414-e6ab811425c6/cni-plugins/0.log"
May 25 07:15:43.812424 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:43.812399    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jwq4_a6aeda4f-547c-488a-9414-e6ab811425c6/bond-cni-plugin/0.log"
May 25 07:15:43.831723 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:43.831696    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jwq4_a6aeda4f-547c-488a-9414-e6ab811425c6/routeoverride-cni/0.log"
May 25 07:15:43.850861 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:43.850821    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jwq4_a6aeda4f-547c-488a-9414-e6ab811425c6/whereabouts-cni-bincopy/0.log"
May 25 07:15:43.870088 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:43.870065    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5jwq4_a6aeda4f-547c-488a-9414-e6ab811425c6/whereabouts-cni/0.log"
May 25 07:15:43.902890 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:43.902859    2572 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-7f8fv/perf-node-gather-daemonset-h4tz4"
May 25 07:15:44.183495 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:44.183467    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lqsp4_28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3/network-metrics-daemon/0.log"
May 25 07:15:44.200439 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:44.200413    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-lqsp4_28839bdc-f2b5-4a95-bc4e-4a2ab9bab5e3/kube-rbac-proxy/0.log"
May 25 07:15:45.032257 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:45.032225    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-controller/0.log"
May 25 07:15:45.048695 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:45.048670    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/0.log"
May 25 07:15:45.057239 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:45.057219    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovn-acl-logging/1.log"
May 25 07:15:45.073697 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:45.073675    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/kube-rbac-proxy-node/0.log"
May 25 07:15:45.092655 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:45.092628    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/kube-rbac-proxy-ovn-metrics/0.log"
May 25 07:15:45.109872 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:45.109830    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/northd/0.log"
May 25 07:15:45.127748 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:45.127716    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/nbdb/0.log"
May 25 07:15:45.146673 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:45.146644    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/sbdb/0.log"
May 25 07:15:45.239903 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:45.239870    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-nxvdb_cddf423c-dbfd-4f24-95d5-dd6089ee0803/ovnkube-controller/0.log"
May 25 07:15:46.466675 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:46.466594    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-pttm7_50545ca0-b117-4f61-8439-62bd12fbf0c0/network-check-target-container/0.log"
May 25 07:15:47.203691 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:47.203662    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-2mdqz_a068851b-398a-41ba-8fec-1afddc1890f5/iptables-alerter/0.log"
May 25 07:15:47.789320 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:47.789243    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-5bxk8_d3930217-a45d-4bff-8456-a261f55e2daa/tuned/0.log"
May 25 07:15:49.225230 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:49.225196    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-697587599-w5dzf_c4147cb5-83db-48f2-8efd-ffc001b1d4c4/cluster-samples-operator/0.log"
May 25 07:15:49.240543 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:49.240515    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-samples-operator_cluster-samples-operator-697587599-w5dzf_c4147cb5-83db-48f2-8efd-ffc001b1d4c4/cluster-samples-operator-watch/0.log"
May 25 07:15:50.041194 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:50.041162    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-57b5b68b8-h9nz4_8d2c6188-c2a6-43cd-a2ec-f7eb401bb061/service-ca-operator/1.log"
May 25 07:15:50.042459 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:50.042421    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-57b5b68b8-h9nz4_8d2c6188-c2a6-43cd-a2ec-f7eb401bb061/service-ca-operator/0.log"
May 25 07:15:50.626596 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:50.626564    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-5bqmd_869a2061-282e-499e-a729-c1df0fc783a3/csi-driver/0.log"
May 25 07:15:50.646570 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:50.646543    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-5bqmd_869a2061-282e-499e-a729-c1df0fc783a3/csi-node-driver-registrar/0.log"
May 25 07:15:50.664822 ip-10-0-128-6 kubenswrapper[2572]: I0525 07:15:50.664796    2572 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-5bqmd_869a2061-282e-499e-a729-c1df0fc783a3/csi-liveness-probe/0.log"