May 26 02:36:02.318221 ip-10-0-139-173 systemd[1]: Starting Kubernetes Kubelet...
May 26 02:36:02.778871 ip-10-0-139-173 kubenswrapper[2551]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 26 02:36:02.778871 ip-10-0-139-173 kubenswrapper[2551]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 26 02:36:02.778871 ip-10-0-139-173 kubenswrapper[2551]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 26 02:36:02.778871 ip-10-0-139-173 kubenswrapper[2551]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 26 02:36:02.778871 ip-10-0-139-173 kubenswrapper[2551]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 26 02:36:02.782310 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.782223    2551 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 26 02:36:02.788699 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788674    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 26 02:36:02.788699 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788693    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 26 02:36:02.788699 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788699    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 26 02:36:02.788699 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788703    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788708    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788712    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788718    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788722    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788726    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788729    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788733    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788737    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788740    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788744    2551 feature_gate.go:328] unrecognized feature gate: Example
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788747    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788751    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788755    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788759    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788763    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788767    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788770    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788774    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 26 02:36:02.788975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788778    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788781    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788790    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788794    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788797    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788801    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788805    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788808    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788812    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788816    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788823    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788828    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788833    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788839    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788843    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788848    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788852    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788856    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788860    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 26 02:36:02.789734 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788864    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788868    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788872    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788878    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788884    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788889    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788893    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788921    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788925    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788930    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788934    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788938    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788943    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788947    2551 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788951    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788955    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788959    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788963    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788967    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 26 02:36:02.790588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788971    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788975    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788979    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788983    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788987    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788992    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.788997    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789001    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789006    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789011    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789015    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789019    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789024    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789028    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789032    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789036    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789040    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789044    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789050    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789055    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 26 02:36:02.791419 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789059    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789063    2551 feature_gate.go:328] unrecognized feature gate: Example2
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789067    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789072    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789076    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789080    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789724    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789733    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789738    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789743    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789747    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789751    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789755    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789759    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789766    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789771    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789776    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789780    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789784    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789788    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 26 02:36:02.792011 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789792    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789797    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789801    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789805    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789810    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789815    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789819    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789823    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789828    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789832    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789837    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789841    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789846    2551 feature_gate.go:328] unrecognized feature gate: Example
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789850    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789854    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789858    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789862    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789866    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789873    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789878    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 26 02:36:02.792556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789884    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789889    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789893    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789917    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789922    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789926    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789931    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789935    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789939    2551 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789943    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789947    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789954    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789960    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789965    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789970    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789975    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789979    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789984    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789988    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 26 02:36:02.793185 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789993    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.789997    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790001    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790005    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790010    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790015    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790019    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790023    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790027    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790031    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790035    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790039    2551 feature_gate.go:328] unrecognized feature gate: Example2
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790044    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790050    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790054    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790058    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790061    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790065    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790070    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790074    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 26 02:36:02.793857 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790079    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790084    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790088    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790092    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790096    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790099    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790103    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790108    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790112    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790116    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790120    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790124    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.790128    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790224    2551 flags.go:64] FLAG: --address="0.0.0.0"
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790234    2551 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790246    2551 flags.go:64] FLAG: --anonymous-auth="true"
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790253    2551 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790262    2551 flags.go:64] FLAG: --authentication-token-webhook="false"
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790267    2551 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790275    2551 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790281    2551 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 26 02:36:02.794377 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790287    2551 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790291    2551 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790298    2551 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790303    2551 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790309    2551 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790314    2551 flags.go:64] FLAG: --cgroup-root=""
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790318    2551 flags.go:64] FLAG: --cgroups-per-qos="true"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790323    2551 flags.go:64] FLAG: --client-ca-file=""
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790328    2551 flags.go:64] FLAG: --cloud-config=""
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790333    2551 flags.go:64] FLAG: --cloud-provider="external"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790342    2551 flags.go:64] FLAG: --cluster-dns="[]"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790350    2551 flags.go:64] FLAG: --cluster-domain=""
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790354    2551 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790359    2551 flags.go:64] FLAG: --config-dir=""
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790364    2551 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790370    2551 flags.go:64] FLAG: --container-log-max-files="5"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790375    2551 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790380    2551 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790385    2551 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790391    2551 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790397    2551 flags.go:64] FLAG: --contention-profiling="false"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790402    2551 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790406    2551 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790412    2551 flags.go:64] FLAG: --cpu-manager-policy="none"
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790416    2551 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 26 02:36:02.795003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790426    2551 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790431    2551 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790436    2551 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790440    2551 flags.go:64] FLAG: --enable-load-reader="false"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790447    2551 flags.go:64] FLAG: --enable-server="true"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790452    2551 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790458    2551 flags.go:64] FLAG: --event-burst="100"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790463    2551 flags.go:64] FLAG: --event-qps="50"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790468    2551 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790473    2551 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790478    2551 flags.go:64] FLAG: --eviction-hard=""
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790490    2551 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790494    2551 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790499    2551 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790504    2551 flags.go:64] FLAG: --eviction-soft=""
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790508    2551 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790513    2551 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790521    2551 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790526    2551 flags.go:64] FLAG: --experimental-mounter-path=""
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790531    2551 flags.go:64] FLAG: --fail-cgroupv1="false"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790535    2551 flags.go:64] FLAG: --fail-swap-on="true"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790540    2551 flags.go:64] FLAG: --feature-gates=""
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790547    2551 flags.go:64] FLAG: --file-check-frequency="20s"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790551    2551 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790556    2551 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 26 02:36:02.795589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790562    2551 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790567    2551 flags.go:64] FLAG: --healthz-port="10248"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790572    2551 flags.go:64] FLAG: --help="false"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790577    2551 flags.go:64] FLAG: --hostname-override="ip-10-0-139-173.ec2.internal"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790582    2551 flags.go:64] FLAG: --housekeeping-interval="10s"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790587    2551 flags.go:64] FLAG: --http-check-frequency="20s"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790592    2551 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790599    2551 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790604    2551 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790609    2551 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790613    2551 flags.go:64] FLAG: --image-service-endpoint=""
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790619    2551 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790624    2551 flags.go:64] FLAG: --kube-api-burst="100"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790630    2551 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790635    2551 flags.go:64] FLAG: --kube-api-qps="50"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790640    2551 flags.go:64] FLAG: --kube-reserved=""
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790645    2551 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790661    2551 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790666    2551 flags.go:64] FLAG: --kubelet-cgroups=""
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790671    2551 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790675    2551 flags.go:64] FLAG: --lock-file=""
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790680    2551 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790686    2551 flags.go:64] FLAG: --log-flush-frequency="5s"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790690    2551 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 26 02:36:02.796214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790701    2551 flags.go:64] FLAG: --log-json-split-stream="false"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790705    2551 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790710    2551 flags.go:64] FLAG: --log-text-split-stream="false"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790715    2551 flags.go:64] FLAG: --logging-format="text"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790719    2551 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790725    2551 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790730    2551 flags.go:64] FLAG: --manifest-url=""
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790734    2551 flags.go:64] FLAG: --manifest-url-header=""
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790741    2551 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790746    2551 flags.go:64] FLAG: --max-open-files="1000000"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790752    2551 flags.go:64] FLAG: --max-pods="110"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790757    2551 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790762    2551 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790767    2551 flags.go:64] FLAG: --memory-manager-policy="None"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790771    2551 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790777    2551 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790783    2551 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790787    2551 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790798    2551 flags.go:64] FLAG: --node-status-max-images="50"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790803    2551 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790808    2551 flags.go:64] FLAG: --oom-score-adj="-999"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790813    2551 flags.go:64] FLAG: --pod-cidr=""
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790818    2551 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 26 02:36:02.796762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790827    2551 flags.go:64] FLAG: --pod-manifest-path=""
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790831    2551 flags.go:64] FLAG: --pod-max-pids="-1"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790836    2551 flags.go:64] FLAG: --pods-per-core="0"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790841    2551 flags.go:64] FLAG: --port="10250"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790846    2551 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790850    2551 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-070fd55118605d407"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790855    2551 flags.go:64] FLAG: --qos-reserved=""
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790860    2551 flags.go:64] FLAG: --read-only-port="10255"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790864    2551 flags.go:64] FLAG: --register-node="true"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790869    2551 flags.go:64] FLAG: --register-schedulable="true"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790875    2551 flags.go:64] FLAG: --register-with-taints=""
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790882    2551 flags.go:64] FLAG: --registry-burst="10"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790886    2551 flags.go:64] FLAG: --registry-qps="5"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790890    2551 flags.go:64] FLAG: --reserved-cpus=""
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790911    2551 flags.go:64] FLAG: --reserved-memory=""
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790918    2551 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790923    2551 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790927    2551 flags.go:64] FLAG: --rotate-certificates="false"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790932    2551 flags.go:64] FLAG: --rotate-server-certificates="false"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790936    2551 flags.go:64] FLAG: --runonce="false"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790941    2551 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790946    2551 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790951    2551 flags.go:64] FLAG: --seccomp-default="false"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790956    2551 flags.go:64] FLAG: --serialize-image-pulls="true"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790965    2551 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790969    2551 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 26 02:36:02.797319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790975    2551 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790980    2551 flags.go:64] FLAG: --storage-driver-password="root"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790984    2551 flags.go:64] FLAG: --storage-driver-secure="false"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790989    2551 flags.go:64] FLAG: --storage-driver-table="stats"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790993    2551 flags.go:64] FLAG: --storage-driver-user="root"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.790999    2551 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791004    2551 flags.go:64] FLAG: --sync-frequency="1m0s"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791009    2551 flags.go:64] FLAG: --system-cgroups=""
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791013    2551 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791022    2551 flags.go:64] FLAG: --system-reserved-cgroup=""
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791027    2551 flags.go:64] FLAG: --tls-cert-file=""
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791032    2551 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791039    2551 flags.go:64] FLAG: --tls-min-version=""
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791044    2551 flags.go:64] FLAG: --tls-private-key-file=""
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791049    2551 flags.go:64] FLAG: --topology-manager-policy="none"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791053    2551 flags.go:64] FLAG: --topology-manager-policy-options=""
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791061    2551 flags.go:64] FLAG: --topology-manager-scope="container"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791066    2551 flags.go:64] FLAG: --v="2"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791073    2551 flags.go:64] FLAG: --version="false"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791079    2551 flags.go:64] FLAG: --vmodule=""
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791085    2551 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.791090    2551 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791240    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791246    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 26 02:36:02.798073 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791251    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791255    2551 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791259    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791263    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791267    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791271    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791277    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791282    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791286    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791290    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791294    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791298    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791302    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791306    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791311    2551 feature_gate.go:328] unrecognized feature gate: Example
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791315    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791320    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791324    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791328    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791333    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 26 02:36:02.799081 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791337    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791340    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791345    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791350    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791356    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791361    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791365    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791369    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791373    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791378    2551 feature_gate.go:328] unrecognized feature gate: Example2
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791382    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791386    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791390    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791394    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791398    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791402    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791406    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791410    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791415    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791419    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 26 02:36:02.800003 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791423    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791427    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791430    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791434    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791438    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791442    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791446    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791453    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791458    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791462    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791466    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791470    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791474    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791478    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791482    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791486    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791492    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791497    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791501    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791505    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 26 02:36:02.800991 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791509    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791513    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791517    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791524    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791530    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791534    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791538    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791542    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791546    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791551    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791555    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791559    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791564    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791568    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791571    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791576    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791580    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791584    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791588    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 26 02:36:02.801599 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791592    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791597    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791601    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791608    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.791613    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.792258    2551 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.799984    2551 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.800004    2551 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800080    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800088    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800094    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800099    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800103    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800108    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800112    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 26 02:36:02.802151 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800116    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800120    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800124    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800129    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800133    2551 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800137    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800141    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800145    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800150    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800154    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800158    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800162    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800166    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800170    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800174    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800178    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800182    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800186    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800191    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800195    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 26 02:36:02.802530 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800199    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800204    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800208    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800213    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800217    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800221    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800225    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800229    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800233    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800237    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800241    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800245    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800249    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800254    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800258    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800263    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800267    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800271    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800275    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800279    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 26 02:36:02.803259 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800283    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800288    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800292    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800296    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800300    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800304    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800308    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800311    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800315    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800318    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800322    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800327    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800330    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800335    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800341    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800347    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800353    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800359    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800364    2551 feature_gate.go:328] unrecognized feature gate: Example2
May 26 02:36:02.803771 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800368    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800373    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800377    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800382    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800385    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800390    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800394    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800398    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800402    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800407    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800411    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800415    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800419    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800424    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800429    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800433    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800437    2551 feature_gate.go:328] unrecognized feature gate: Example
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800441    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800446    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 26 02:36:02.804371 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800450    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.800457    2551 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800614    2551 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800622    2551 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800627    2551 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800632    2551 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800637    2551 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800642    2551 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800646    2551 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800650    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800654    2551 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800658    2551 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800662    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800667    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800671    2551 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 26 02:36:02.804821 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800675    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800679    2551 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800683    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800687    2551 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800691    2551 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800695    2551 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800699    2551 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800703    2551 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800707    2551 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800711    2551 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800715    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800720    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800724    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800728    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800732    2551 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800736    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800740    2551 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800744    2551 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800748    2551 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800752    2551 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 26 02:36:02.805408 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800780    2551 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800786    2551 feature_gate.go:328] unrecognized feature gate: DualReplica
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800791    2551 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800795    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800801    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800805    2551 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800809    2551 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800813    2551 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800817    2551 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800821    2551 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800825    2551 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800829    2551 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800833    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800837    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800841    2551 feature_gate.go:328] unrecognized feature gate: NewOLM
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800845    2551 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800849    2551 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800853    2551 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800857    2551 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800861    2551 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 26 02:36:02.806215 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800865    2551 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800869    2551 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800874    2551 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800878    2551 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800882    2551 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800886    2551 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800890    2551 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800894    2551 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800915    2551 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800920    2551 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800923    2551 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800927    2551 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800930    2551 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800934    2551 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800937    2551 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800943    2551 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800950    2551 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800956    2551 feature_gate.go:328] unrecognized feature gate: Example2
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800960    2551 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 26 02:36:02.806778 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800963    2551 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800967    2551 feature_gate.go:328] unrecognized feature gate: Example
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800971    2551 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800975    2551 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800979    2551 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800982    2551 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800986    2551 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800990    2551 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800994    2551 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.800998    2551 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.801002    2551 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.801006    2551 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.801010    2551 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:02.801016    2551 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.801023    2551 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 26 02:36:02.807312 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.801883    2551 server.go:962] "Client rotation is on, will bootstrap in background"
May 26 02:36:02.807686 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.806711    2551 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 26 02:36:02.807812 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.807799    2551 server.go:1019] "Starting client certificate rotation"
May 26 02:36:02.807932 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.807914    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 26 02:36:02.807971 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.807951    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 26 02:36:02.833827 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.833800    2551 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 26 02:36:02.835987 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.835964    2551 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 26 02:36:02.853144 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.853122    2551 log.go:25] "Validated CRI v1 runtime API"
May 26 02:36:02.858316 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.858299    2551 log.go:25] "Validated CRI v1 image API"
May 26 02:36:02.859595 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.859581    2551 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 26 02:36:02.865057 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.865036    2551 fs.go:135] Filesystem UUIDs: map[0491a351-b4f4-4e72-9c83-069be810119b:/dev/nvme0n1p4 4986dd42-0021-482f-94f4-66bcc05d7fe9:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2]
May 26 02:36:02.865136 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.865056    2551 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 26 02:36:02.865622 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.865607    2551 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 26 02:36:02.869284 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.869178    2551 manager.go:217] Machine: {Timestamp:2026-05-26 02:36:02.868664394 +0000 UTC m=+0.425453072 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3125148 MemoryCapacity:33164496896 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d31f896ac5c90d66315b377c277bb SystemUUID:ec2d31f8-96ac-5c90-d663-15b377c277bb BootID:bccb4945-4153-4dd2-b79a-4bd7d13a2a37 Filesystems:[{Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16582246400 Type:vfs Inodes:4048400 HasInodes:true} {Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6632902656 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16582250496 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:4a:01:a1:e8:db Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:4a:01:a1:e8:db Speed:0 Mtu:9001} {Name:ovs-system MacAddress:66:7a:47:10:78:ef Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:33164496896 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:37486592 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 26 02:36:02.870044 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.870032    2551 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 26 02:36:02.870126 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.870115    2551 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 26 02:36:02.872156 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.872133    2551 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 26 02:36:02.872302 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.872160    2551 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-139-173.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 26 02:36:02.872349 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.872312    2551 topology_manager.go:138] "Creating topology manager with none policy"
May 26 02:36:02.872349 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.872320    2551 container_manager_linux.go:306] "Creating device plugin manager"
May 26 02:36:02.872349 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.872328    2551 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 26 02:36:02.872349 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.872343    2551 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 26 02:36:02.873157 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.873147    2551 state_mem.go:36] "Initialized new in-memory state store"
May 26 02:36:02.873270 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.873261    2551 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 26 02:36:02.875907 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.875886    2551 kubelet.go:491] "Attempting to sync node with API server"
May 26 02:36:02.875942 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.875912    2551 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 26 02:36:02.875942 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.875926    2551 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 26 02:36:02.875942 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.875935    2551 kubelet.go:397] "Adding apiserver pod source"
May 26 02:36:02.876042 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.875944    2551 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 26 02:36:02.877492 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.877481    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 26 02:36:02.877543 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.877500    2551 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 26 02:36:02.881130 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.881114    2551 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 26 02:36:02.881332 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.881317    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8zk5m"
May 26 02:36:02.883076 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.883063    2551 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 26 02:36:02.885389 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.885377    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 26 02:36:02.885433 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.885394    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 26 02:36:02.885433 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.885401    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 26 02:36:02.885433 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.885406    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 26 02:36:02.885433 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.885420    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 26 02:36:02.885433 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.885426    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 26 02:36:02.885433 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.885432    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 26 02:36:02.885631 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.885439    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 26 02:36:02.885631 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.885445    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 26 02:36:02.885631 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.885451    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 26 02:36:02.885631 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.885483    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 26 02:36:02.885631 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.885489    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 26 02:36:02.886460 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.886450    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 26 02:36:02.886460 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.886460    2551 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 26 02:36:02.889038 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.889021    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-8zk5m"
May 26 02:36:02.889591 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.889570    2551 csi_plugin.go:988] Failed to contact API server when waiting for CSINode publishing: csinodes.storage.k8s.io "ip-10-0-139-173.ec2.internal" is forbidden: User "system:anonymous" cannot get resource "csinodes" in API group "storage.k8s.io" at the cluster scope
May 26 02:36:02.889668 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:02.889572    2551 reflector.go:200] "Failed to watch" err="failed to list *v1.Node: nodes \"ip-10-0-139-173.ec2.internal\" is forbidden: User \"system:anonymous\" cannot list resource \"nodes\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Node"
May 26 02:36:02.889981 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:02.889964    2551 reflector.go:200] "Failed to watch" err="failed to list *v1.Service: services is forbidden: User \"system:anonymous\" cannot list resource \"services\" in API group \"\" at the cluster scope" logger="UnhandledError" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service"
May 26 02:36:02.890054 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.890044    2551 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 26 02:36:02.890105 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.890080    2551 server.go:1295] "Started kubelet"
May 26 02:36:02.890165 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.890138    2551 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 26 02:36:02.894118 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.894071    2551 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 26 02:36:02.894177 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.894140    2551 server_v1.go:47] "podresources" method="list" useActivePods=true
May 26 02:36:02.894393 ip-10-0-139-173 systemd[1]: Started Kubernetes Kubelet.
May 26 02:36:02.896233 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.896215    2551 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 26 02:36:02.898130 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.898116    2551 server.go:317] "Adding debug handlers to kubelet server"
May 26 02:36:02.902850 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:02.902830    2551 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 26 02:36:02.903052 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903037    2551 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 26 02:36:02.903115 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903049    2551 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 26 02:36:02.903115 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903090    2551 volume_manager.go:295] "The desired_state_of_world populator starts"
May 26 02:36:02.903115 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903098    2551 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 26 02:36:02.903285 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903266    2551 reconstruct.go:97] "Volume reconstruction finished"
May 26 02:36:02.903335 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903287    2551 reconciler.go:26] "Reconciler: start to sync state"
May 26 02:36:02.903335 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903306    2551 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 26 02:36:02.903603 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903589    2551 factory.go:55] Registering systemd factory
May 26 02:36:02.903665 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903607    2551 factory.go:223] Registration of the systemd container factory successfully
May 26 02:36:02.903845 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903830    2551 factory.go:153] Registering CRI-O factory
May 26 02:36:02.903930 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903848    2551 factory.go:223] Registration of the crio container factory successfully
May 26 02:36:02.903930 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903893    2551 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 26 02:36:02.903930 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903927    2551 factory.go:103] Registering Raw factory
May 26 02:36:02.904072 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.903941    2551 manager.go:1196] Started watching for new ooms in manager
May 26 02:36:02.904072 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:02.903974    2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:02.904287 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.904270    2551 manager.go:319] Starting recovery of all containers
May 26 02:36:02.906592 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.906564    2551 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 26 02:36:02.913885 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.913731    2551 manager.go:324] Recovery completed
May 26 02:36:02.915453 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:02.915437    2551 watcher.go:152] Failed to watch directory "/sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service": inotify_add_watch /sys/fs/cgroup/system.slice/systemd-update-utmp-runlevel.service: no such file or directory
May 26 02:36:02.917055 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:02.917038    2551 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-139-173.ec2.internal\" not found" node="ip-10-0-139-173.ec2.internal"
May 26 02:36:02.918210 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.918199    2551 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 26 02:36:02.920067 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.919948    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
May 26 02:36:02.920067 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.919981    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
May 26 02:36:02.920067 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.919995    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
May 26 02:36:02.920462 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.920450    2551 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 26 02:36:02.920517 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.920461    2551 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 26 02:36:02.920517 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.920479    2551 state_mem.go:36] "Initialized new in-memory state store"
May 26 02:36:02.922955 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.922943    2551 policy_none.go:49] "None policy: Start"
May 26 02:36:02.923000 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.922960    2551 memory_manager.go:186] "Starting memorymanager" policy="None"
May 26 02:36:02.923000 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.922969    2551 state_mem.go:35] "Initializing new in-memory state store"
May 26 02:36:02.961547 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.961529    2551 manager.go:341] "Starting Device Plugin manager"
May 26 02:36:02.976087 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:02.961576    2551 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 26 02:36:02.976087 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.961590    2551 server.go:85] "Starting device plugin registration server"
May 26 02:36:02.976087 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.961823    2551 eviction_manager.go:189] "Eviction manager: starting control loop"
May 26 02:36:02.976087 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.961836    2551 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 26 02:36:02.976087 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.961956    2551 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 26 02:36:02.976087 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.962038    2551 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 26 02:36:02.976087 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:02.962046    2551 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 26 02:36:02.976087 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:02.962590    2551 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 26 02:36:02.976087 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:02.962634    2551 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:03.039012 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.038931    2551 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 26 02:36:03.040019 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.040005    2551 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 26 02:36:03.040092 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.040032    2551 status_manager.go:230] "Starting to sync pod status with apiserver"
May 26 02:36:03.040092 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.040051    2551 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 26 02:36:03.040092 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.040060    2551 kubelet.go:2451] "Starting kubelet main sync loop"
May 26 02:36:03.040198 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.040098    2551 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 26 02:36:03.042407 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.042386    2551 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 26 02:36:03.062419 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.062396    2551 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 26 02:36:03.063476 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.063461    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
May 26 02:36:03.063574 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.063491    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
May 26 02:36:03.063574 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.063505    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
May 26 02:36:03.063574 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.063530    2551 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-139-173.ec2.internal"
May 26 02:36:03.074781 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.074757    2551 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-139-173.ec2.internal"
May 26 02:36:03.074781 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.074782    2551 kubelet_node_status.go:597] "Error updating node status, will retry" err="error getting node \"ip-10-0-139-173.ec2.internal\": node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:03.102494 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.102469    2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:03.140725 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.140689    2551 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"]
May 26 02:36:03.140820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.140799    2551 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 26 02:36:03.143147 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.143130    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
May 26 02:36:03.143221 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.143160    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
May 26 02:36:03.143221 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.143169    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
May 26 02:36:03.144549 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.144538    2551 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 26 02:36:03.144724 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.144711    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.144764 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.144741    2551 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 26 02:36:03.145210 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.145195    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
May 26 02:36:03.145210 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.145203    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
May 26 02:36:03.145312 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.145225    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
May 26 02:36:03.145312 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.145228    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
May 26 02:36:03.145312 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.145235    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
May 26 02:36:03.145312 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.145241    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
May 26 02:36:03.146310 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.146293    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.146377 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.146327    2551 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 26 02:36:03.147004 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.146968    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientMemory"
May 26 02:36:03.147004 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.146995    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasNoDiskPressure"
May 26 02:36:03.147004 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.147006    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeHasSufficientPID"
May 26 02:36:03.173427 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.173402    2551 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-173.ec2.internal\" not found" node="ip-10-0-139-173.ec2.internal"
May 26 02:36:03.177861 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.177846    2551 kubelet.go:3336] "No need to create a mirror pod, since failed to get node info from the cluster" err="node \"ip-10-0-139-173.ec2.internal\" not found" node="ip-10-0-139-173.ec2.internal"
May 26 02:36:03.202947 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.202927    2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:03.205168 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.205152    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9efcfc9ee347a3d38d55c694ea6e537c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal\" (UID: \"9efcfc9ee347a3d38d55c694ea6e537c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.205240 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.205180    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9efcfc9ee347a3d38d55c694ea6e537c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal\" (UID: \"9efcfc9ee347a3d38d55c694ea6e537c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.205240 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.205205    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8c67b16bff298a6f27ca11c5dc88021f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-173.ec2.internal\" (UID: \"8c67b16bff298a6f27ca11c5dc88021f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.303492 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.303410    2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:03.305644 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.305624    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9efcfc9ee347a3d38d55c694ea6e537c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal\" (UID: \"9efcfc9ee347a3d38d55c694ea6e537c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.305739 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.305658    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9efcfc9ee347a3d38d55c694ea6e537c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal\" (UID: \"9efcfc9ee347a3d38d55c694ea6e537c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.305739 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.305688    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8c67b16bff298a6f27ca11c5dc88021f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-173.ec2.internal\" (UID: \"8c67b16bff298a6f27ca11c5dc88021f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.305739 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.305726    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/9efcfc9ee347a3d38d55c694ea6e537c-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal\" (UID: \"9efcfc9ee347a3d38d55c694ea6e537c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.305859 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.305741    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/9efcfc9ee347a3d38d55c694ea6e537c-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal\" (UID: \"9efcfc9ee347a3d38d55c694ea6e537c\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.305859 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.305742    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/8c67b16bff298a6f27ca11c5dc88021f-config\") pod \"kube-apiserver-proxy-ip-10-0-139-173.ec2.internal\" (UID: \"8c67b16bff298a6f27ca11c5dc88021f\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.404048 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.404021    2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:03.476358 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.476332    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.480893 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.480875    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"
May 26 02:36:03.504724 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.504699    2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:03.605199 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.605121    2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:03.705619 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.705590    2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:03.806438 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.806409    2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:03.807522 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.807503    2551 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 26 02:36:03.807666 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.807649    2551 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 26 02:36:03.807725 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.807671    2551 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 26 02:36:03.896447 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.896399    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-25 02:31:02 +0000 UTC" deadline="2027-12-17 14:51:00.956692427 +0000 UTC"
May 26 02:36:03.896447 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.896444    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="13692h14m57.060253273s"
May 26 02:36:03.903929 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.903909    2551 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 26 02:36:03.906776 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:03.906759    2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:03.914626 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.914609    2551 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 26 02:36:03.934143 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.934123    2551 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-cr2ht"
May 26 02:36:03.941610 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.941588    2551 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-cr2ht"
May 26 02:36:03.959761 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:03.959730    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9efcfc9ee347a3d38d55c694ea6e537c.slice/crio-429b20aeae0c020becb0877f2f22385894c8406da18875793e7da53a75c3a577 WatchSource:0}: Error finding container 429b20aeae0c020becb0877f2f22385894c8406da18875793e7da53a75c3a577: Status 404 returned error can't find the container with id 429b20aeae0c020becb0877f2f22385894c8406da18875793e7da53a75c3a577
May 26 02:36:03.960017 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:03.959991    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8c67b16bff298a6f27ca11c5dc88021f.slice/crio-95d4acf230a0766330fb49c8ba7f81fe5404d38dce38cda17dd6dd7ef70a190d WatchSource:0}: Error finding container 95d4acf230a0766330fb49c8ba7f81fe5404d38dce38cda17dd6dd7ef70a190d: Status 404 returned error can't find the container with id 95d4acf230a0766330fb49c8ba7f81fe5404d38dce38cda17dd6dd7ef70a190d
May 26 02:36:03.965271 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:03.965257    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 26 02:36:04.007666 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:04.007634    2551 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-139-173.ec2.internal\" not found"
May 26 02:36:04.042953 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.042893    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal" event={"ID":"9efcfc9ee347a3d38d55c694ea6e537c","Type":"ContainerStarted","Data":"429b20aeae0c020becb0877f2f22385894c8406da18875793e7da53a75c3a577"}
May 26 02:36:04.043763 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.043741    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal" event={"ID":"8c67b16bff298a6f27ca11c5dc88021f","Type":"ContainerStarted","Data":"95d4acf230a0766330fb49c8ba7f81fe5404d38dce38cda17dd6dd7ef70a190d"}
May 26 02:36:04.059532 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.059512    2551 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 26 02:36:04.103510 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.103484    2551 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal"
May 26 02:36:04.114616 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.114556    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 26 02:36:04.116825 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.116813    2551 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal"
May 26 02:36:04.124743 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.124727    2551 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 26 02:36:04.386437 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.386359    2551 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 26 02:36:04.878022 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.877946    2551 apiserver.go:52] "Watching apiserver"
May 26 02:36:04.884171 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.884150    2551 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 26 02:36:04.886767 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.886745    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-multus/network-metrics-daemon-29jmf","openshift-network-diagnostics/network-check-target-wzhws","openshift-network-operator/iptables-alerter-4w92x","kube-system/konnectivity-agent-vk2g6","kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal","openshift-multus/multus-gmx7w","openshift-ovn-kubernetes/ovnkube-node-t8q5v","openshift-cluster-node-tuning-operator/tuned-sq9nq","openshift-dns/node-resolver-j55gs","openshift-image-registry/node-ca-q8dr4","openshift-multus/multus-additional-cni-plugins-5x5dt"]
May 26 02:36:04.888415 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.888367    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.891218 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.891197    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 26 02:36:04.891322 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.891307    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 26 02:36:04.891947 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.891441    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4w92x"
May 26 02:36:04.892052 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.891989    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-hx9km\""
May 26 02:36:04.893481 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.893460    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 26 02:36:04.893591 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.893578    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:04.893674 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:04.893651    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:04.893763 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.893743    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-9sxmd\""
May 26 02:36:04.894006 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.893988    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 26 02:36:04.894121 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.893994    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 26 02:36:04.894845 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.894823    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vk2g6"
May 26 02:36:04.896112 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.896068    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:04.896214 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.896205    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.896982 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.896750    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 26 02:36:04.896982 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.896776    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-tlg8v\""
May 26 02:36:04.897100 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.897031    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 26 02:36:04.897891 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.897872    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.898569 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.898433    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 26 02:36:04.898969 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.898668    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 26 02:36:04.899289 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.899266    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-rlgkh\""
May 26 02:36:04.899386 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.899371    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 26 02:36:04.899617 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.899598    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-tqht4\""
May 26 02:36:04.900006 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.899799    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 26 02:36:04.900006 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.899823    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 26 02:36:04.900006 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.899939    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 26 02:36:04.900006 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.899982    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 26 02:36:04.900298 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.900227    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 26 02:36:04.900298 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.900285    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 26 02:36:04.900707 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.900406    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 26 02:36:04.900707 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.900514    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 26 02:36:04.900707 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.900600    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:04.900707 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:04.900697    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:04.900934 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.900732    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 26 02:36:04.900992 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.900952    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 26 02:36:04.901227 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.901208    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-4hv88\""
May 26 02:36:04.902134 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.902115    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q8dr4"
May 26 02:36:04.903871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.903743    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j55gs"
May 26 02:36:04.903871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.903793    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:04.904347 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.904329    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 26 02:36:04.904426 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.904390    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 26 02:36:04.904619 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.904585    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 26 02:36:04.904619 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.904609    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-9c9f8\""
May 26 02:36:04.906051 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.906030    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-sv2nk\""
May 26 02:36:04.906257 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.906241    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 26 02:36:04.906556 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.906539    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 26 02:36:04.906657 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.906624    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 26 02:36:04.906730 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.906682    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-gcvxv\""
May 26 02:36:04.906785 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.906744    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 26 02:36:04.914109 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914092    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-etc-selinux\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:04.914199 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914116    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-run-ovn\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.914199 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914133    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e26384c-7c3d-4dad-bab9-a74ec1259042-os-release\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:04.914199 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914158    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-modprobe-d\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.914352 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914181    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-sys-fs\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:04.914352 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914227    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-multus-conf-dir\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.914352 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914304    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-run-openvswitch\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.914352 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914330    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nd54s\" (UniqueName: \"kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s\") pod \"network-check-target-wzhws\" (UID: \"53dbc3bb-daa3-4d5c-a004-f5046a91b1ae\") " pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:04.914538 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914354    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d82f5f80-58cd-4e40-a66e-27301ac76a37-konnectivity-ca\") pod \"konnectivity-agent-vk2g6\" (UID: \"d82f5f80-58cd-4e40-a66e-27301ac76a37\") " pod="kube-system/konnectivity-agent-vk2g6"
May 26 02:36:04.914538 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914380    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-multus-socket-dir-parent\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.914538 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914410    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zq45h\" (UniqueName: \"kubernetes.io/projected/f2d576e5-b71f-438e-8dde-aebb68efe86e-kube-api-access-zq45h\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:04.914538 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914432    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-run-k8s-cni-cncf-io\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.914538 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914491    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-run-systemd\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.914538 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914524    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-prvs2\" (UniqueName: \"kubernetes.io/projected/7dd95e00-cc75-4517-a4cb-2484530f0fbd-kube-api-access-prvs2\") pod \"node-ca-q8dr4\" (UID: \"7dd95e00-cc75-4517-a4cb-2484530f0fbd\") " pod="openshift-image-registry/node-ca-q8dr4"
May 26 02:36:04.914818 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914558    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-sysctl-d\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.914818 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914598    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-var-lib-kubelet\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.914818 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914633    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-tuned\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.914818 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914677    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7q9gz\" (UniqueName: \"kubernetes.io/projected/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-kube-api-access-7q9gz\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:04.914818 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914700    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-var-lib-kubelet\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.914818 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914727    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-etc-kubernetes\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.914818 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914749    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-zl9x7\" (UniqueName: \"kubernetes.io/projected/8df2d22d-804e-42c6-9100-d32e0f707aae-kube-api-access-zl9x7\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.914818 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914770    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-slash\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.914818 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914791    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-var-lib-openvswitch\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.914818 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914811    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-node-log\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914834    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914855    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7cef8ed3-f979-4a27-983b-8f89cfa6b7ae-hosts-file\") pod \"node-resolver-j55gs\" (UID: \"7cef8ed3-f979-4a27-983b-8f89cfa6b7ae\") " pod="openshift-dns/node-resolver-j55gs"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914915    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-os-release\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.914943    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8df2d22d-804e-42c6-9100-d32e0f707aae-cni-binary-copy\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915010    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e26384c-7c3d-4dad-bab9-a74ec1259042-cnibin\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915032    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-kubernetes\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915067    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-sysctl-conf\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915100    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ffa480f-baaf-4a18-adf0-dd6ed1d2915a-host-slash\") pod \"iptables-alerter-4w92x\" (UID: \"5ffa480f-baaf-4a18-adf0-dd6ed1d2915a\") " pod="openshift-network-operator/iptables-alerter-4w92x"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915127    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915169    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-systemd-units\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915206    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac847df6-96a0-478b-8c22-796d6809971a-env-overrides\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915227    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac847df6-96a0-478b-8c22-796d6809971a-ovn-node-metrics-cert\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.915280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915260    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7dd95e00-cc75-4517-a4cb-2484530f0fbd-serviceca\") pod \"node-ca-q8dr4\" (UID: \"7dd95e00-cc75-4517-a4cb-2484530f0fbd\") " pod="openshift-image-registry/node-ca-q8dr4"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915292    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5e26384c-7c3d-4dad-bab9-a74ec1259042-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915314    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5e26384c-7c3d-4dad-bab9-a74ec1259042-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915357    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-lib-modules\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915392    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pbggm\" (UniqueName: \"kubernetes.io/projected/7cef8ed3-f979-4a27-983b-8f89cfa6b7ae-kube-api-access-pbggm\") pod \"node-resolver-j55gs\" (UID: \"7cef8ed3-f979-4a27-983b-8f89cfa6b7ae\") " pod="openshift-dns/node-resolver-j55gs"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915434    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915456    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e26384c-7c3d-4dad-bab9-a74ec1259042-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915489    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tpvpj\" (UniqueName: \"kubernetes.io/projected/5e26384c-7c3d-4dad-bab9-a74ec1259042-kube-api-access-tpvpj\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915531    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nl7jp\" (UniqueName: \"kubernetes.io/projected/c578cb33-6d42-4ae0-a636-58b4be832e7d-kube-api-access-nl7jp\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915572    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-registration-dir\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915593    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-cnibin\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915611    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-etc-openvswitch\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915633    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-run-ovn-kubernetes\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915658    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-cni-netd\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915683    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-run-netns\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915702    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac847df6-96a0-478b-8c22-796d6809971a-ovnkube-config\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.915850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915718    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-device-dir\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915755    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cef8ed3-f979-4a27-983b-8f89cfa6b7ae-tmp-dir\") pod \"node-resolver-j55gs\" (UID: \"7cef8ed3-f979-4a27-983b-8f89cfa6b7ae\") " pod="openshift-dns/node-resolver-j55gs"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915777    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-system-cni-dir\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915802    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-var-lib-cni-multus\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915824    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-hostroot\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915857    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8df2d22d-804e-42c6-9100-d32e0f707aae-multus-daemon-config\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915883    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-sysconfig\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915923    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-sys\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915949    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d82f5f80-58cd-4e40-a66e-27301ac76a37-agent-certs\") pod \"konnectivity-agent-vk2g6\" (UID: \"d82f5f80-58cd-4e40-a66e-27301ac76a37\") " pod="kube-system/konnectivity-agent-vk2g6"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915971    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-kubelet\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.915992    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-host\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.916017    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c578cb33-6d42-4ae0-a636-58b4be832e7d-tmp\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.916090    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ffa480f-baaf-4a18-adf0-dd6ed1d2915a-iptables-alerter-script\") pod \"iptables-alerter-4w92x\" (UID: \"5ffa480f-baaf-4a18-adf0-dd6ed1d2915a\") " pod="openshift-network-operator/iptables-alerter-4w92x"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.916408    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-multus-cni-dir\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.916449    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-run-multus-certs\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.916492    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac847df6-96a0-478b-8c22-796d6809971a-ovnkube-script-lib\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.916625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.916515    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7dd95e00-cc75-4517-a4cb-2484530f0fbd-host\") pod \"node-ca-q8dr4\" (UID: \"7dd95e00-cc75-4517-a4cb-2484530f0fbd\") " pod="openshift-image-registry/node-ca-q8dr4"
May 26 02:36:04.917415 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.916641    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e26384c-7c3d-4dad-bab9-a74ec1259042-system-cni-dir\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:04.917415 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.916702    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-run\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.917415 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.916948    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-socket-dir\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:04.917415 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.917022    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-var-lib-cni-bin\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.917415 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.917081    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-log-socket\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.917415 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.917321    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e26384c-7c3d-4dad-bab9-a74ec1259042-cni-binary-copy\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:04.917843 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.917821    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-run-netns\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:04.917945 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.917870    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-cni-bin\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.917945 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.917923    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bn77q\" (UniqueName: \"kubernetes.io/projected/ac847df6-96a0-478b-8c22-796d6809971a-kube-api-access-bn77q\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:04.918049 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.917961    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-systemd\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:04.918049 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.917994    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4kcg4\" (UniqueName: \"kubernetes.io/projected/5ffa480f-baaf-4a18-adf0-dd6ed1d2915a-kube-api-access-4kcg4\") pod \"iptables-alerter-4w92x\" (UID: \"5ffa480f-baaf-4a18-adf0-dd6ed1d2915a\") " pod="openshift-network-operator/iptables-alerter-4w92x"
May 26 02:36:04.942285 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.942259    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-25 02:31:03 +0000 UTC" deadline="2028-03-01 20:05:33.100585124 +0000 UTC"
May 26 02:36:04.942285 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:04.942285    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="15497h29m28.158304074s"
May 26 02:36:05.004306 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.004284    2551 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 26 02:36:05.018223 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018197    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-socket-dir\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.018370 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018228    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-var-lib-cni-bin\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.018370 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018247    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-log-socket\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.018370 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018271    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e26384c-7c3d-4dad-bab9-a74ec1259042-cni-binary-copy\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.018370 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018300    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-run-netns\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.018370 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018325    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-cni-bin\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.018370 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018335    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-var-lib-cni-bin\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.018370 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018340    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-log-socket\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018347    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bn77q\" (UniqueName: \"kubernetes.io/projected/ac847df6-96a0-478b-8c22-796d6809971a-kube-api-access-bn77q\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018380    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-run-netns\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018390    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-cni-bin\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018392    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-socket-dir\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018410    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-systemd\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018440    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4kcg4\" (UniqueName: \"kubernetes.io/projected/5ffa480f-baaf-4a18-adf0-dd6ed1d2915a-kube-api-access-4kcg4\") pod \"iptables-alerter-4w92x\" (UID: \"5ffa480f-baaf-4a18-adf0-dd6ed1d2915a\") " pod="openshift-network-operator/iptables-alerter-4w92x"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018468    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-etc-selinux\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018496    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-run-ovn\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018501    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-systemd\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018608    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-etc-selinux\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018620    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e26384c-7c3d-4dad-bab9-a74ec1259042-os-release\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018629    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-run-ovn\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018657    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-modprobe-d\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018681    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-sys-fs\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018686    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/5e26384c-7c3d-4dad-bab9-a74ec1259042-os-release\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.018716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018713    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-multus-conf-dir\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018730    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-sys-fs\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018738    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-run-openvswitch\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018767    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-multus-conf-dir\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018766    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-modprobe-d\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018763    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nd54s\" (UniqueName: \"kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s\") pod \"network-check-target-wzhws\" (UID: \"53dbc3bb-daa3-4d5c-a004-f5046a91b1ae\") " pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018798    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-run-openvswitch\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018806    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d82f5f80-58cd-4e40-a66e-27301ac76a37-konnectivity-ca\") pod \"konnectivity-agent-vk2g6\" (UID: \"d82f5f80-58cd-4e40-a66e-27301ac76a37\") " pod="kube-system/konnectivity-agent-vk2g6"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018844    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-multus-socket-dir-parent\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018865    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/5e26384c-7c3d-4dad-bab9-a74ec1259042-cni-binary-copy\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018869    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zq45h\" (UniqueName: \"kubernetes.io/projected/f2d576e5-b71f-438e-8dde-aebb68efe86e-kube-api-access-zq45h\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018918    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-run-k8s-cni-cncf-io\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018943    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-run-systemd\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018945    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-multus-socket-dir-parent\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018967    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-prvs2\" (UniqueName: \"kubernetes.io/projected/7dd95e00-cc75-4517-a4cb-2484530f0fbd-kube-api-access-prvs2\") pod \"node-ca-q8dr4\" (UID: \"7dd95e00-cc75-4517-a4cb-2484530f0fbd\") " pod="openshift-image-registry/node-ca-q8dr4"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018957    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-run-k8s-cni-cncf-io\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018992    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-sysctl-d\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.019427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.018993    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-run-systemd\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019098    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-var-lib-kubelet\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019126    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-tuned\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019151    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7q9gz\" (UniqueName: \"kubernetes.io/projected/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-kube-api-access-7q9gz\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019153    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-var-lib-kubelet\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019174    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-var-lib-kubelet\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019182    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-sysctl-d\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019198    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-etc-kubernetes\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019217    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-zl9x7\" (UniqueName: \"kubernetes.io/projected/8df2d22d-804e-42c6-9100-d32e0f707aae-kube-api-access-zl9x7\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019227    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-var-lib-kubelet\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019234    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-slash\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019267    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-var-lib-openvswitch\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019264    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-slash\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019296    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-node-log\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019325    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019347    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7cef8ed3-f979-4a27-983b-8f89cfa6b7ae-hosts-file\") pod \"node-resolver-j55gs\" (UID: \"7cef8ed3-f979-4a27-983b-8f89cfa6b7ae\") " pod="openshift-dns/node-resolver-j55gs"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019369    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-os-release\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019381    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-var-lib-openvswitch\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.020173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019356    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/d82f5f80-58cd-4e40-a66e-27301ac76a37-konnectivity-ca\") pod \"konnectivity-agent-vk2g6\" (UID: \"d82f5f80-58cd-4e40-a66e-27301ac76a37\") " pod="kube-system/konnectivity-agent-vk2g6"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019348    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-node-log\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019428    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/7cef8ed3-f979-4a27-983b-8f89cfa6b7ae-hosts-file\") pod \"node-resolver-j55gs\" (UID: \"7cef8ed3-f979-4a27-983b-8f89cfa6b7ae\") " pod="openshift-dns/node-resolver-j55gs"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019415    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8df2d22d-804e-42c6-9100-d32e0f707aae-cni-binary-copy\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019471    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-os-release\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019476    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e26384c-7c3d-4dad-bab9-a74ec1259042-cnibin\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019472    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-etc-kubernetes\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019511    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-kubernetes\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019542    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-sysctl-conf\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019556    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/5e26384c-7c3d-4dad-bab9-a74ec1259042-cnibin\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019567    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ffa480f-baaf-4a18-adf0-dd6ed1d2915a-host-slash\") pod \"iptables-alerter-4w92x\" (UID: \"5ffa480f-baaf-4a18-adf0-dd6ed1d2915a\") " pod="openshift-network-operator/iptables-alerter-4w92x"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019604    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019614    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/5ffa480f-baaf-4a18-adf0-dd6ed1d2915a-host-slash\") pod \"iptables-alerter-4w92x\" (UID: \"5ffa480f-baaf-4a18-adf0-dd6ed1d2915a\") " pod="openshift-network-operator/iptables-alerter-4w92x"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019560    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-kubernetes\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019629    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-systemd-units\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:05.019633    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019653    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac847df6-96a0-478b-8c22-796d6809971a-env-overrides\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019671    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-systemd-units\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.020871 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019677    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-kubelet-dir\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019626    2551 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019682    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-sysctl-conf\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:05.019710    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs podName:9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8 nodeName:}" failed. No retries permitted until 2026-05-26 02:36:05.519674654 +0000 UTC m=+3.076463332 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs") pod "network-metrics-daemon-29jmf" (UID: "9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019733    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac847df6-96a0-478b-8c22-796d6809971a-ovn-node-metrics-cert\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019761    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7dd95e00-cc75-4517-a4cb-2484530f0fbd-serviceca\") pod \"node-ca-q8dr4\" (UID: \"7dd95e00-cc75-4517-a4cb-2484530f0fbd\") " pod="openshift-image-registry/node-ca-q8dr4"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019791    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5e26384c-7c3d-4dad-bab9-a74ec1259042-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019826    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/8df2d22d-804e-42c6-9100-d32e0f707aae-cni-binary-copy\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019854    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5e26384c-7c3d-4dad-bab9-a74ec1259042-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019890    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-lib-modules\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019946    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pbggm\" (UniqueName: \"kubernetes.io/projected/7cef8ed3-f979-4a27-983b-8f89cfa6b7ae-kube-api-access-pbggm\") pod \"node-resolver-j55gs\" (UID: \"7cef8ed3-f979-4a27-983b-8f89cfa6b7ae\") " pod="openshift-dns/node-resolver-j55gs"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.019976    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020010    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e26384c-7c3d-4dad-bab9-a74ec1259042-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020041    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tpvpj\" (UniqueName: \"kubernetes.io/projected/5e26384c-7c3d-4dad-bab9-a74ec1259042-kube-api-access-tpvpj\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020072    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nl7jp\" (UniqueName: \"kubernetes.io/projected/c578cb33-6d42-4ae0-a636-58b4be832e7d-kube-api-access-nl7jp\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020098    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-registration-dir\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.021684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020127    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-cnibin\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020164    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-etc-openvswitch\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020188    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-run-ovn-kubernetes\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020220    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/ac847df6-96a0-478b-8c22-796d6809971a-env-overrides\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020225    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-cni-netd\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020288    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-run-netns\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020314    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac847df6-96a0-478b-8c22-796d6809971a-ovnkube-config\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020339    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-device-dir\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020344    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/5e26384c-7c3d-4dad-bab9-a74ec1259042-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020364    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cef8ed3-f979-4a27-983b-8f89cfa6b7ae-tmp-dir\") pod \"node-resolver-j55gs\" (UID: \"7cef8ed3-f979-4a27-983b-8f89cfa6b7ae\") " pod="openshift-dns/node-resolver-j55gs"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020401    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-system-cni-dir\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020405    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/5e26384c-7c3d-4dad-bab9-a74ec1259042-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020445    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-var-lib-cni-multus\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020472    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-hostroot\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020497    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8df2d22d-804e-42c6-9100-d32e0f707aae-multus-daemon-config\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020508    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020524    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-sysconfig\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.022408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020582    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-sysconfig\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020642    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/5e26384c-7c3d-4dad-bab9-a74ec1259042-tuning-conf-dir\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020682    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-sys\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020694    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-lib-modules\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020712    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d82f5f80-58cd-4e40-a66e-27301ac76a37-agent-certs\") pod \"konnectivity-agent-vk2g6\" (UID: \"d82f5f80-58cd-4e40-a66e-27301ac76a37\") " pod="kube-system/konnectivity-agent-vk2g6"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020737    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-kubelet\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020746    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-sys\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020768    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-host\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020795    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-system-cni-dir\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020806    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-var-lib-cni-multus\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020793    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c578cb33-6d42-4ae0-a636-58b4be832e7d-tmp\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020845    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-device-dir\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020868    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ffa480f-baaf-4a18-adf0-dd6ed1d2915a-iptables-alerter-script\") pod \"iptables-alerter-4w92x\" (UID: \"5ffa480f-baaf-4a18-adf0-dd6ed1d2915a\") " pod="openshift-network-operator/iptables-alerter-4w92x"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020917    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-multus-cni-dir\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020946    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-run-multus-certs\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021127    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/7cef8ed3-f979-4a27-983b-8f89cfa6b7ae-tmp-dir\") pod \"node-resolver-j55gs\" (UID: \"7cef8ed3-f979-4a27-983b-8f89cfa6b7ae\") " pod="openshift-dns/node-resolver-j55gs"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021197    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/8df2d22d-804e-42c6-9100-d32e0f707aae-multus-daemon-config\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021200    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-etc-openvswitch\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.023179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021205    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/ac847df6-96a0-478b-8c22-796d6809971a-ovnkube-config\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021242    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-run-ovn-kubernetes\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021263    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-run-netns\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021272    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-cni-netd\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021307    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-cnibin\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021330    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/f2d576e5-b71f-438e-8dde-aebb68efe86e-registration-dir\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020838    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-hostroot\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020970    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac847df6-96a0-478b-8c22-796d6809971a-ovnkube-script-lib\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021465    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7dd95e00-cc75-4517-a4cb-2484530f0fbd-host\") pod \"node-ca-q8dr4\" (UID: \"7dd95e00-cc75-4517-a4cb-2484530f0fbd\") " pod="openshift-image-registry/node-ca-q8dr4"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021492    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e26384c-7c3d-4dad-bab9-a74ec1259042-system-cni-dir\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021515    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-run\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021588    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-run\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021646    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-multus-cni-dir\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021692    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/8df2d22d-804e-42c6-9100-d32e0f707aae-host-run-multus-certs\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021760    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/7dd95e00-cc75-4517-a4cb-2484530f0fbd-host\") pod \"node-ca-q8dr4\" (UID: \"7dd95e00-cc75-4517-a4cb-2484530f0fbd\") " pod="openshift-image-registry/node-ca-q8dr4"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021820    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/ac847df6-96a0-478b-8c22-796d6809971a-host-kubelet\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021840    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/5ffa480f-baaf-4a18-adf0-dd6ed1d2915a-iptables-alerter-script\") pod \"iptables-alerter-4w92x\" (UID: \"5ffa480f-baaf-4a18-adf0-dd6ed1d2915a\") " pod="openshift-network-operator/iptables-alerter-4w92x"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.020681    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/7dd95e00-cc75-4517-a4cb-2484530f0fbd-serviceca\") pod \"node-ca-q8dr4\" (UID: \"7dd95e00-cc75-4517-a4cb-2484530f0fbd\") " pod="openshift-image-registry/node-ca-q8dr4"
May 26 02:36:05.023952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021937    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/c578cb33-6d42-4ae0-a636-58b4be832e7d-host\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.024766 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.021943    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/5e26384c-7c3d-4dad-bab9-a74ec1259042-system-cni-dir\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.024766 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.022168    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/ac847df6-96a0-478b-8c22-796d6809971a-ovnkube-script-lib\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.024766 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.023382    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/c578cb33-6d42-4ae0-a636-58b4be832e7d-etc-tuned\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.024766 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.023678    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/ac847df6-96a0-478b-8c22-796d6809971a-ovn-node-metrics-cert\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.024766 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.023919    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/c578cb33-6d42-4ae0-a636-58b4be832e7d-tmp\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.024766 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.024070    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/d82f5f80-58cd-4e40-a66e-27301ac76a37-agent-certs\") pod \"konnectivity-agent-vk2g6\" (UID: \"d82f5f80-58cd-4e40-a66e-27301ac76a37\") " pod="kube-system/konnectivity-agent-vk2g6"
May 26 02:36:05.025084 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:05.024800    2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 02:36:05.025084 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:05.024822    2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 02:36:05.025084 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:05.024833    2551 projected.go:194] Error preparing data for projected volume kube-api-access-nd54s for pod openshift-network-diagnostics/network-check-target-wzhws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 02:36:05.025084 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:05.024880    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s podName:53dbc3bb-daa3-4d5c-a004-f5046a91b1ae nodeName:}" failed. No retries permitted until 2026-05-26 02:36:05.524866535 +0000 UTC m=+3.081655217 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-nd54s" (UniqueName: "kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s") pod "network-check-target-wzhws" (UID: "53dbc3bb-daa3-4d5c-a004-f5046a91b1ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 02:36:05.027375 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.027305    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zq45h\" (UniqueName: \"kubernetes.io/projected/f2d576e5-b71f-438e-8dde-aebb68efe86e-kube-api-access-zq45h\") pod \"aws-ebs-csi-driver-node-md5xq\" (UID: \"f2d576e5-b71f-438e-8dde-aebb68efe86e\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.027484 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.027415    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bn77q\" (UniqueName: \"kubernetes.io/projected/ac847df6-96a0-478b-8c22-796d6809971a-kube-api-access-bn77q\") pod \"ovnkube-node-t8q5v\" (UID: \"ac847df6-96a0-478b-8c22-796d6809971a\") " pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.027847 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.027823    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-prvs2\" (UniqueName: \"kubernetes.io/projected/7dd95e00-cc75-4517-a4cb-2484530f0fbd-kube-api-access-prvs2\") pod \"node-ca-q8dr4\" (UID: \"7dd95e00-cc75-4517-a4cb-2484530f0fbd\") " pod="openshift-image-registry/node-ca-q8dr4"
May 26 02:36:05.031598 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.029476    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4kcg4\" (UniqueName: \"kubernetes.io/projected/5ffa480f-baaf-4a18-adf0-dd6ed1d2915a-kube-api-access-4kcg4\") pod \"iptables-alerter-4w92x\" (UID: \"5ffa480f-baaf-4a18-adf0-dd6ed1d2915a\") " pod="openshift-network-operator/iptables-alerter-4w92x"
May 26 02:36:05.031598 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.029484    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7q9gz\" (UniqueName: \"kubernetes.io/projected/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-kube-api-access-7q9gz\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:05.031598 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.029576    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-zl9x7\" (UniqueName: \"kubernetes.io/projected/8df2d22d-804e-42c6-9100-d32e0f707aae-kube-api-access-zl9x7\") pod \"multus-gmx7w\" (UID: \"8df2d22d-804e-42c6-9100-d32e0f707aae\") " pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.031598 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.030771    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pbggm\" (UniqueName: \"kubernetes.io/projected/7cef8ed3-f979-4a27-983b-8f89cfa6b7ae-kube-api-access-pbggm\") pod \"node-resolver-j55gs\" (UID: \"7cef8ed3-f979-4a27-983b-8f89cfa6b7ae\") " pod="openshift-dns/node-resolver-j55gs"
May 26 02:36:05.031598 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.031099    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nl7jp\" (UniqueName: \"kubernetes.io/projected/c578cb33-6d42-4ae0-a636-58b4be832e7d-kube-api-access-nl7jp\") pod \"tuned-sq9nq\" (UID: \"c578cb33-6d42-4ae0-a636-58b4be832e7d\") " pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.032977 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.032890    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tpvpj\" (UniqueName: \"kubernetes.io/projected/5e26384c-7c3d-4dad-bab9-a74ec1259042-kube-api-access-tpvpj\") pod \"multus-additional-cni-plugins-5x5dt\" (UID: \"5e26384c-7c3d-4dad-bab9-a74ec1259042\") " pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.072155 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.072133    2551 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 26 02:36:05.203612 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.203528    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-sq9nq"
May 26 02:36:05.212418 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.212381    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-4w92x"
May 26 02:36:05.222003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.221981    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-vk2g6"
May 26 02:36:05.228327 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.228307    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq"
May 26 02:36:05.235849 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.235830    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-gmx7w"
May 26 02:36:05.243412 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.243393    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:05.249991 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.249968    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-q8dr4"
May 26 02:36:05.251221 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:05.251200    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf2d576e5_b71f_438e_8dde_aebb68efe86e.slice/crio-5cb17a6c4fe3e056bddedca49ce405e14bec6ab94a42a0b8b3fd8681abd11f11 WatchSource:0}: Error finding container 5cb17a6c4fe3e056bddedca49ce405e14bec6ab94a42a0b8b3fd8681abd11f11: Status 404 returned error can't find the container with id 5cb17a6c4fe3e056bddedca49ce405e14bec6ab94a42a0b8b3fd8681abd11f11
May 26 02:36:05.253617 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:05.253597    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5ffa480f_baaf_4a18_adf0_dd6ed1d2915a.slice/crio-388ebbd3766586f4b3708d0cab4a8299cf315b3a2c10c30bd59c4393b3a5303d WatchSource:0}: Error finding container 388ebbd3766586f4b3708d0cab4a8299cf315b3a2c10c30bd59c4393b3a5303d: Status 404 returned error can't find the container with id 388ebbd3766586f4b3708d0cab4a8299cf315b3a2c10c30bd59c4393b3a5303d
May 26 02:36:05.257140 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.257122    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-j55gs"
May 26 02:36:05.263734 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.263716    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-5x5dt"
May 26 02:36:05.264556 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:05.264532    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8df2d22d_804e_42c6_9100_d32e0f707aae.slice/crio-189072e90fcc4f787685156767869370c905d885dee028c90c612b9a0a30c86a WatchSource:0}: Error finding container 189072e90fcc4f787685156767869370c905d885dee028c90c612b9a0a30c86a: Status 404 returned error can't find the container with id 189072e90fcc4f787685156767869370c905d885dee028c90c612b9a0a30c86a
May 26 02:36:05.265388 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:05.265370    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podc578cb33_6d42_4ae0_a636_58b4be832e7d.slice/crio-fcb1a74bd03294256aab98d630e3f776e7a84439dd25ca873f8bb2e9b5a09744 WatchSource:0}: Error finding container fcb1a74bd03294256aab98d630e3f776e7a84439dd25ca873f8bb2e9b5a09744: Status 404 returned error can't find the container with id fcb1a74bd03294256aab98d630e3f776e7a84439dd25ca873f8bb2e9b5a09744
May 26 02:36:05.266393 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:05.266370    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd82f5f80_58cd_4e40_a66e_27301ac76a37.slice/crio-01146c00c8ea33de9df332fd78e38daef09dbcd3054225e3016deab33790669e WatchSource:0}: Error finding container 01146c00c8ea33de9df332fd78e38daef09dbcd3054225e3016deab33790669e: Status 404 returned error can't find the container with id 01146c00c8ea33de9df332fd78e38daef09dbcd3054225e3016deab33790669e
May 26 02:36:05.273566 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:05.273545    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac847df6_96a0_478b_8c22_796d6809971a.slice/crio-c05135820692529973b43b7f89e3c4db4cb9c82d0a838c93a69492a89ed6a53d WatchSource:0}: Error finding container c05135820692529973b43b7f89e3c4db4cb9c82d0a838c93a69492a89ed6a53d: Status 404 returned error can't find the container with id c05135820692529973b43b7f89e3c4db4cb9c82d0a838c93a69492a89ed6a53d
May 26 02:36:05.277268 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:05.277239    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7dd95e00_cc75_4517_a4cb_2484530f0fbd.slice/crio-0e280d54906dfdd94f0f62d7a876e1f59d8011d53cbfdf114c0198ddd5f4c3d7 WatchSource:0}: Error finding container 0e280d54906dfdd94f0f62d7a876e1f59d8011d53cbfdf114c0198ddd5f4c3d7: Status 404 returned error can't find the container with id 0e280d54906dfdd94f0f62d7a876e1f59d8011d53cbfdf114c0198ddd5f4c3d7
May 26 02:36:05.278221 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:05.278196    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7cef8ed3_f979_4a27_983b_8f89cfa6b7ae.slice/crio-969b8ccc80edbaaff63bf27f96fd9b654603d187a10700923989a6e2cf41a6d9 WatchSource:0}: Error finding container 969b8ccc80edbaaff63bf27f96fd9b654603d187a10700923989a6e2cf41a6d9: Status 404 returned error can't find the container with id 969b8ccc80edbaaff63bf27f96fd9b654603d187a10700923989a6e2cf41a6d9
May 26 02:36:05.279617 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:05.279600    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5e26384c_7c3d_4dad_bab9_a74ec1259042.slice/crio-1b75f92fc8994ecbfc4f5a98e621dfe082abd91b3867f99b387da68a72e97595 WatchSource:0}: Error finding container 1b75f92fc8994ecbfc4f5a98e621dfe082abd91b3867f99b387da68a72e97595: Status 404 returned error can't find the container with id 1b75f92fc8994ecbfc4f5a98e621dfe082abd91b3867f99b387da68a72e97595
May 26 02:36:05.300299 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.300279    2551 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 26 02:36:05.525651 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.525536    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nd54s\" (UniqueName: \"kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s\") pod \"network-check-target-wzhws\" (UID: \"53dbc3bb-daa3-4d5c-a004-f5046a91b1ae\") " pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:05.525651 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.525584    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:05.525859 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:05.525699    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 02:36:05.525859 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:05.525715    2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 02:36:05.525859 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:05.525745    2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 02:36:05.525859 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:05.525759    2551 projected.go:194] Error preparing data for projected volume kube-api-access-nd54s for pod openshift-network-diagnostics/network-check-target-wzhws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 02:36:05.525859 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:05.525769    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs podName:9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8 nodeName:}" failed. No retries permitted until 2026-05-26 02:36:06.525750621 +0000 UTC m=+4.082539283 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs") pod "network-metrics-daemon-29jmf" (UID: "9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 02:36:05.525859 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:05.525809    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s podName:53dbc3bb-daa3-4d5c-a004-f5046a91b1ae nodeName:}" failed. No retries permitted until 2026-05-26 02:36:06.52579351 +0000 UTC m=+4.082582156 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-nd54s" (UniqueName: "kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s") pod "network-check-target-wzhws" (UID: "53dbc3bb-daa3-4d5c-a004-f5046a91b1ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 02:36:05.942913 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.942793    2551 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-25 02:31:03 +0000 UTC" deadline="2027-10-27 15:21:15.346585044 +0000 UTC"
May 26 02:36:05.942913 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:05.942833    2551 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12468h45m9.403756235s"
May 26 02:36:06.074912 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.074858    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal" event={"ID":"8c67b16bff298a6f27ca11c5dc88021f","Type":"ContainerStarted","Data":"14dffad85c0ca31552a0f814647fc01229c9fbbcede788da561e2d07dba81e38"}
May 26 02:36:06.087876 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.087813    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-139-173.ec2.internal" podStartSLOduration=2.087794305 podStartE2EDuration="2.087794305s" podCreationTimestamp="2026-05-26 02:36:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:36:06.087777366 +0000 UTC m=+3.644566056" watchObservedRunningTime="2026-05-26 02:36:06.087794305 +0000 UTC m=+3.644582971"
May 26 02:36:06.097870 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.097836    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j55gs" event={"ID":"7cef8ed3-f979-4a27-983b-8f89cfa6b7ae","Type":"ContainerStarted","Data":"969b8ccc80edbaaff63bf27f96fd9b654603d187a10700923989a6e2cf41a6d9"}
May 26 02:36:06.120027 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.119997    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q8dr4" event={"ID":"7dd95e00-cc75-4517-a4cb-2484530f0fbd","Type":"ContainerStarted","Data":"0e280d54906dfdd94f0f62d7a876e1f59d8011d53cbfdf114c0198ddd5f4c3d7"}
May 26 02:36:06.134740 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.134710    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sq9nq" event={"ID":"c578cb33-6d42-4ae0-a636-58b4be832e7d","Type":"ContainerStarted","Data":"fcb1a74bd03294256aab98d630e3f776e7a84439dd25ca873f8bb2e9b5a09744"}
May 26 02:36:06.153258 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.153223    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq" event={"ID":"f2d576e5-b71f-438e-8dde-aebb68efe86e","Type":"ContainerStarted","Data":"5cb17a6c4fe3e056bddedca49ce405e14bec6ab94a42a0b8b3fd8681abd11f11"}
May 26 02:36:06.170598 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.170401    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5x5dt" event={"ID":"5e26384c-7c3d-4dad-bab9-a74ec1259042","Type":"ContainerStarted","Data":"1b75f92fc8994ecbfc4f5a98e621dfe082abd91b3867f99b387da68a72e97595"}
May 26 02:36:06.172377 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.172311    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v" event={"ID":"ac847df6-96a0-478b-8c22-796d6809971a","Type":"ContainerStarted","Data":"c05135820692529973b43b7f89e3c4db4cb9c82d0a838c93a69492a89ed6a53d"}
May 26 02:36:06.178063 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.178013    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vk2g6" event={"ID":"d82f5f80-58cd-4e40-a66e-27301ac76a37","Type":"ContainerStarted","Data":"01146c00c8ea33de9df332fd78e38daef09dbcd3054225e3016deab33790669e"}
May 26 02:36:06.186878 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.186844    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gmx7w" event={"ID":"8df2d22d-804e-42c6-9100-d32e0f707aae","Type":"ContainerStarted","Data":"189072e90fcc4f787685156767869370c905d885dee028c90c612b9a0a30c86a"}
May 26 02:36:06.199794 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.199702    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4w92x" event={"ID":"5ffa480f-baaf-4a18-adf0-dd6ed1d2915a","Type":"ContainerStarted","Data":"388ebbd3766586f4b3708d0cab4a8299cf315b3a2c10c30bd59c4393b3a5303d"}
May 26 02:36:06.535606 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.535509    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:06.535606 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:06.535593    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nd54s\" (UniqueName: \"kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s\") pod \"network-check-target-wzhws\" (UID: \"53dbc3bb-daa3-4d5c-a004-f5046a91b1ae\") " pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:06.535851 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:06.535735    2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 02:36:06.535851 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:06.535753    2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 02:36:06.535851 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:06.535765    2551 projected.go:194] Error preparing data for projected volume kube-api-access-nd54s for pod openshift-network-diagnostics/network-check-target-wzhws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 02:36:06.535851 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:06.535821    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s podName:53dbc3bb-daa3-4d5c-a004-f5046a91b1ae nodeName:}" failed. No retries permitted until 2026-05-26 02:36:08.535803228 +0000 UTC m=+6.092591875 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-nd54s" (UniqueName: "kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s") pod "network-check-target-wzhws" (UID: "53dbc3bb-daa3-4d5c-a004-f5046a91b1ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 02:36:06.536254 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:06.536237    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 02:36:06.536323 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:06.536292    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs podName:9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8 nodeName:}" failed. No retries permitted until 2026-05-26 02:36:08.536277028 +0000 UTC m=+6.093065680 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs") pod "network-metrics-daemon-29jmf" (UID: "9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 02:36:07.043704 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:07.043013    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:07.043704 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:07.043137    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:07.043704 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:07.043238    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:07.043704 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:07.043323    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:08.213475 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:08.213365    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal" event={"ID":"9efcfc9ee347a3d38d55c694ea6e537c","Type":"ContainerStarted","Data":"5f34d8e02c56a41c325b87d1dd71522c64be23fcb2d28ba9bba4b05a5bfd84ae"}
May 26 02:36:08.555968 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:08.555152    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nd54s\" (UniqueName: \"kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s\") pod \"network-check-target-wzhws\" (UID: \"53dbc3bb-daa3-4d5c-a004-f5046a91b1ae\") " pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:08.555968 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:08.555210    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:08.555968 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:08.555356    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 02:36:08.555968 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:08.555415    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs podName:9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8 nodeName:}" failed. No retries permitted until 2026-05-26 02:36:12.555396196 +0000 UTC m=+10.112184862 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs") pod "network-metrics-daemon-29jmf" (UID: "9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 02:36:08.555968 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:08.555803    2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 02:36:08.555968 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:08.555822    2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 02:36:08.555968 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:08.555835    2551 projected.go:194] Error preparing data for projected volume kube-api-access-nd54s for pod openshift-network-diagnostics/network-check-target-wzhws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 02:36:08.555968 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:08.555877    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s podName:53dbc3bb-daa3-4d5c-a004-f5046a91b1ae nodeName:}" failed. No retries permitted until 2026-05-26 02:36:12.555863066 +0000 UTC m=+10.112651718 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-nd54s" (UniqueName: "kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s") pod "network-check-target-wzhws" (UID: "53dbc3bb-daa3-4d5c-a004-f5046a91b1ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 02:36:09.041668 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:09.040964    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:09.041668 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:09.040999    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:09.041668 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:09.041098    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:09.041668 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:09.041563    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:10.219565 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:10.219479    2551 generic.go:358] "Generic (PLEG): container finished" podID="9efcfc9ee347a3d38d55c694ea6e537c" containerID="5f34d8e02c56a41c325b87d1dd71522c64be23fcb2d28ba9bba4b05a5bfd84ae" exitCode=0
May 26 02:36:10.219565 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:10.219536    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal" event={"ID":"9efcfc9ee347a3d38d55c694ea6e537c","Type":"ContainerDied","Data":"5f34d8e02c56a41c325b87d1dd71522c64be23fcb2d28ba9bba4b05a5bfd84ae"}
May 26 02:36:11.041217 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:11.041153    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:11.041396 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:11.041279    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:11.041762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:11.041739    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:11.041880 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:11.041857    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:12.589759 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:12.589638    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nd54s\" (UniqueName: \"kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s\") pod \"network-check-target-wzhws\" (UID: \"53dbc3bb-daa3-4d5c-a004-f5046a91b1ae\") " pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:12.589759 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:12.589754    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:12.590167 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:12.589868    2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 02:36:12.590167 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:12.589894    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 02:36:12.590167 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:12.589976    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs podName:9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8 nodeName:}" failed. No retries permitted until 2026-05-26 02:36:20.589956083 +0000 UTC m=+18.146744745 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs") pod "network-metrics-daemon-29jmf" (UID: "9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 02:36:12.590408 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:12.589912    2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 02:36:12.590408 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:12.590305    2551 projected.go:194] Error preparing data for projected volume kube-api-access-nd54s for pod openshift-network-diagnostics/network-check-target-wzhws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 02:36:12.590408 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:12.590364    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s podName:53dbc3bb-daa3-4d5c-a004-f5046a91b1ae nodeName:}" failed. No retries permitted until 2026-05-26 02:36:20.590348811 +0000 UTC m=+18.147137463 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-nd54s" (UniqueName: "kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s") pod "network-check-target-wzhws" (UID: "53dbc3bb-daa3-4d5c-a004-f5046a91b1ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 02:36:13.043981 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:13.042890    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:13.043981 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:13.043031    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:13.044781 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:13.044499    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:13.044781 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:13.044606    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:15.040973 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:15.040933    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:15.041398 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:15.041075    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:15.041398 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:15.041129    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:15.041398 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:15.041248    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:17.041111 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:17.041082    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:17.041648 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:17.041189    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:17.041648 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:17.041270    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:17.041648 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:17.041365    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:19.040508 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:19.040469    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:19.040508 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:19.040508    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:19.041454 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:19.040609    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:19.041454 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:19.040738    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:20.650549 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:20.650510    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nd54s\" (UniqueName: \"kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s\") pod \"network-check-target-wzhws\" (UID: \"53dbc3bb-daa3-4d5c-a004-f5046a91b1ae\") " pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:20.651051 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:20.650575    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:20.651051 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:20.650698    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 02:36:20.651051 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:20.650714    2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 26 02:36:20.651051 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:20.650737    2551 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 26 02:36:20.651051 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:20.650750    2551 projected.go:194] Error preparing data for projected volume kube-api-access-nd54s for pod openshift-network-diagnostics/network-check-target-wzhws: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 02:36:20.651051 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:20.650777    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs podName:9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8 nodeName:}" failed. No retries permitted until 2026-05-26 02:36:36.650756189 +0000 UTC m=+34.207544835 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs") pod "network-metrics-daemon-29jmf" (UID: "9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 26 02:36:20.651051 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:20.650798    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s podName:53dbc3bb-daa3-4d5c-a004-f5046a91b1ae nodeName:}" failed. No retries permitted until 2026-05-26 02:36:36.650787846 +0000 UTC m=+34.207576497 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-nd54s" (UniqueName: "kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s") pod "network-check-target-wzhws" (UID: "53dbc3bb-daa3-4d5c-a004-f5046a91b1ae") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 26 02:36:21.041226 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:21.041138    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:21.041370 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:21.041268    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:21.041370 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:21.041322    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:21.041503 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:21.041466    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:23.041629 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.041608    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:23.041963 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:23.041712    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:23.041963 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.041775    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:23.041963 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:23.041859    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:23.241299 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.241274    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal" event={"ID":"9efcfc9ee347a3d38d55c694ea6e537c","Type":"ContainerStarted","Data":"4fbbc4ab7796fc2e6bbbfcda5b11f64f81433d3b9658ee33f8d3cb9f898a85d5"}
May 26 02:36:23.242384 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.242364    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-j55gs" event={"ID":"7cef8ed3-f979-4a27-983b-8f89cfa6b7ae","Type":"ContainerStarted","Data":"20501d856aadce27459d1092dcda4e9de69072f7e90cc9a56829fde74fa5cb78"}
May 26 02:36:23.243716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.243694    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-q8dr4" event={"ID":"7dd95e00-cc75-4517-a4cb-2484530f0fbd","Type":"ContainerStarted","Data":"47523ca56959df97f5f2e245e97b7821c212ce487e979508782694bef5ab4a76"}
May 26 02:36:23.245098 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.245075    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-sq9nq" event={"ID":"c578cb33-6d42-4ae0-a636-58b4be832e7d","Type":"ContainerStarted","Data":"0e3083381a540ca019072f4feb5c90edf3a442eab391bada21e42a7838c00846"}
May 26 02:36:23.246302 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.246283    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq" event={"ID":"f2d576e5-b71f-438e-8dde-aebb68efe86e","Type":"ContainerStarted","Data":"3a596a78fdb4b83981ac783a4f86bbe9f04e0a3fc9465d9f8a20424733fe2d39"}
May 26 02:36:23.247386 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.247363    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5x5dt" event={"ID":"5e26384c-7c3d-4dad-bab9-a74ec1259042","Type":"ContainerStarted","Data":"bd83f4a85f8222778a4f326fba3235872cf119c95996a3727b587bdd62eec817"}
May 26 02:36:23.248759 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.248739    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v" event={"ID":"ac847df6-96a0-478b-8c22-796d6809971a","Type":"ContainerStarted","Data":"fa332139628a699274a6b6f72d0dd52e423fc587037810eef86f7c97b1d218f4"}
May 26 02:36:23.248850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.248763    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v" event={"ID":"ac847df6-96a0-478b-8c22-796d6809971a","Type":"ContainerStarted","Data":"79af497992d0162670892ea7b21f80d42b302eb822e07b5a8b629894463f3b59"}
May 26 02:36:23.250113 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.250012    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-vk2g6" event={"ID":"d82f5f80-58cd-4e40-a66e-27301ac76a37","Type":"ContainerStarted","Data":"4e6aa64509d26daa4739dcc737416e7490b3e54d9c880568a4bf46d4006ce891"}
May 26 02:36:23.251353 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.251335    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-gmx7w" event={"ID":"8df2d22d-804e-42c6-9100-d32e0f707aae","Type":"ContainerStarted","Data":"0110a7bb3d41e0a61da198dad31657c7cb1435270c9263995bfc0a63b518c6b3"}
May 26 02:36:23.254552 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.254509    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-139-173.ec2.internal" podStartSLOduration=19.254499632 podStartE2EDuration="19.254499632s" podCreationTimestamp="2026-05-26 02:36:04 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:36:23.254391695 +0000 UTC m=+20.811180365" watchObservedRunningTime="2026-05-26 02:36:23.254499632 +0000 UTC m=+20.811288301"
May 26 02:36:23.287498 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.286190    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-j55gs" podStartSLOduration=2.778083561 podStartE2EDuration="20.286147814s" podCreationTimestamp="2026-05-26 02:36:03 +0000 UTC" firstStartedPulling="2026-05-26 02:36:05.279973253 +0000 UTC m=+2.836761899" lastFinishedPulling="2026-05-26 02:36:22.788037492 +0000 UTC m=+20.344826152" observedRunningTime="2026-05-26 02:36:23.268318276 +0000 UTC m=+20.825106952" watchObservedRunningTime="2026-05-26 02:36:23.286147814 +0000 UTC m=+20.842936483"
May 26 02:36:23.301493 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.301442    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-vk2g6" podStartSLOduration=10.390156495 podStartE2EDuration="20.301405478s" podCreationTimestamp="2026-05-26 02:36:03 +0000 UTC" firstStartedPulling="2026-05-26 02:36:05.270571311 +0000 UTC m=+2.827359957" lastFinishedPulling="2026-05-26 02:36:15.181820285 +0000 UTC m=+12.738608940" observedRunningTime="2026-05-26 02:36:23.301352442 +0000 UTC m=+20.858141111" watchObservedRunningTime="2026-05-26 02:36:23.301405478 +0000 UTC m=+20.858194145"
May 26 02:36:23.302047 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.301989    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-sq9nq" podStartSLOduration=2.528986756 podStartE2EDuration="20.301975492s" podCreationTimestamp="2026-05-26 02:36:03 +0000 UTC" firstStartedPulling="2026-05-26 02:36:05.267851084 +0000 UTC m=+2.824639745" lastFinishedPulling="2026-05-26 02:36:23.040839832 +0000 UTC m=+20.597628481" observedRunningTime="2026-05-26 02:36:23.286476708 +0000 UTC m=+20.843265378" watchObservedRunningTime="2026-05-26 02:36:23.301975492 +0000 UTC m=+20.858764162"
May 26 02:36:23.318285 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.318249    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-gmx7w" podStartSLOduration=2.725791083 podStartE2EDuration="20.318235847s" podCreationTimestamp="2026-05-26 02:36:03 +0000 UTC" firstStartedPulling="2026-05-26 02:36:05.267500639 +0000 UTC m=+2.824289285" lastFinishedPulling="2026-05-26 02:36:22.85994539 +0000 UTC m=+20.416734049" observedRunningTime="2026-05-26 02:36:23.318054135 +0000 UTC m=+20.874842823" watchObservedRunningTime="2026-05-26 02:36:23.318235847 +0000 UTC m=+20.875024516"
May 26 02:36:23.331598 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:23.331556    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-q8dr4" podStartSLOduration=2.822520111 podStartE2EDuration="20.331545624s" podCreationTimestamp="2026-05-26 02:36:03 +0000 UTC" firstStartedPulling="2026-05-26 02:36:05.279050267 +0000 UTC m=+2.835838915" lastFinishedPulling="2026-05-26 02:36:22.788075774 +0000 UTC m=+20.344864428" observedRunningTime="2026-05-26 02:36:23.331088534 +0000 UTC m=+20.887877196" watchObservedRunningTime="2026-05-26 02:36:23.331545624 +0000 UTC m=+20.888334292"
May 26 02:36:24.222916 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.222757    2551 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 26 02:36:24.254750 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.254722    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq" event={"ID":"f2d576e5-b71f-438e-8dde-aebb68efe86e","Type":"ContainerStarted","Data":"d2c7758f721be0495702165800aa1e9fac1d3cd6db4581982a91431ca254bfed"}
May 26 02:36:24.255931 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.255895    2551 generic.go:358] "Generic (PLEG): container finished" podID="5e26384c-7c3d-4dad-bab9-a74ec1259042" containerID="bd83f4a85f8222778a4f326fba3235872cf119c95996a3727b587bdd62eec817" exitCode=0
May 26 02:36:24.256015 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.255970    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5x5dt" event={"ID":"5e26384c-7c3d-4dad-bab9-a74ec1259042","Type":"ContainerDied","Data":"bd83f4a85f8222778a4f326fba3235872cf119c95996a3727b587bdd62eec817"}
May 26 02:36:24.258456 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.258439    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 02:36:24.258784 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.258758    2551 generic.go:358] "Generic (PLEG): container finished" podID="ac847df6-96a0-478b-8c22-796d6809971a" containerID="fa332139628a699274a6b6f72d0dd52e423fc587037810eef86f7c97b1d218f4" exitCode=1
May 26 02:36:24.258872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.258785    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v" event={"ID":"ac847df6-96a0-478b-8c22-796d6809971a","Type":"ContainerDied","Data":"fa332139628a699274a6b6f72d0dd52e423fc587037810eef86f7c97b1d218f4"}
May 26 02:36:24.258872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.258808    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v" event={"ID":"ac847df6-96a0-478b-8c22-796d6809971a","Type":"ContainerStarted","Data":"d68f17a94a17c062bd8cad5590f04c13e5ab9f8c676a133f5d5f2734a1c99e1f"}
May 26 02:36:24.258872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.258817    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v" event={"ID":"ac847df6-96a0-478b-8c22-796d6809971a","Type":"ContainerStarted","Data":"c083c020e58cfb75d05346b548dc00ccc935e9c47815c606a0e7e206c255ee3e"}
May 26 02:36:24.258872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.258825    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v" event={"ID":"ac847df6-96a0-478b-8c22-796d6809971a","Type":"ContainerStarted","Data":"7ba636a6dbafdab49d0bb4a1316a232a97633432e3c44773701d63bc91224776"}
May 26 02:36:24.258872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.258838    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v" event={"ID":"ac847df6-96a0-478b-8c22-796d6809971a","Type":"ContainerStarted","Data":"8c0105db2bcf33adbed6741ba868c7ab866193bdf9bc81b29cf504a3ee190cdd"}
May 26 02:36:24.260093 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.260070    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-4w92x" event={"ID":"5ffa480f-baaf-4a18-adf0-dd6ed1d2915a","Type":"ContainerStarted","Data":"3fe7e89359590617b4f052dd01119c880b9e5cd8c5db967a79141b081ca9e9c1"}
May 26 02:36:24.971247 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.971105    2551 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-26T02:36:24.222894599Z","UUID":"d1faae7c-8acc-46e6-8084-8eccfa95dd9b","Handler":null,"Name":"","Endpoint":""}
May 26 02:36:24.972822 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.972793    2551 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 26 02:36:24.972822 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:24.972820    2551 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 26 02:36:25.041275 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:25.041249    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:25.041457 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:25.041288    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:25.041524 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:25.041450    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:25.041581 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:25.041536    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:25.263796 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:25.263724    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq" event={"ID":"f2d576e5-b71f-438e-8dde-aebb68efe86e","Type":"ContainerStarted","Data":"4608f35556e951e8cfbfbe92776d526d47dcdcec6adb569f6f5fb87ce0cb1085"}
May 26 02:36:25.280427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:25.280381    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-md5xq" podStartSLOduration=2.454495112 podStartE2EDuration="22.280368841s" podCreationTimestamp="2026-05-26 02:36:03 +0000 UTC" firstStartedPulling="2026-05-26 02:36:05.252869036 +0000 UTC m=+2.809657687" lastFinishedPulling="2026-05-26 02:36:25.078742764 +0000 UTC m=+22.635531416" observedRunningTime="2026-05-26 02:36:25.279751995 +0000 UTC m=+22.836540663" watchObservedRunningTime="2026-05-26 02:36:25.280368841 +0000 UTC m=+22.837157516"
May 26 02:36:25.280976 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:25.280948    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-4w92x" podStartSLOduration=4.691004083 podStartE2EDuration="22.280940146s" podCreationTimestamp="2026-05-26 02:36:03 +0000 UTC" firstStartedPulling="2026-05-26 02:36:05.262340623 +0000 UTC m=+2.819129286" lastFinishedPulling="2026-05-26 02:36:22.852276702 +0000 UTC m=+20.409065349" observedRunningTime="2026-05-26 02:36:24.286383973 +0000 UTC m=+21.843172643" watchObservedRunningTime="2026-05-26 02:36:25.280940146 +0000 UTC m=+22.837728813"
May 26 02:36:26.268917 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:26.268694    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 02:36:26.269363 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:26.269284    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v" event={"ID":"ac847df6-96a0-478b-8c22-796d6809971a","Type":"ContainerStarted","Data":"166c2371391cc0e60603aad7ecd16bd74710185eda6fb90c66d5a83d377837f0"}
May 26 02:36:26.813985 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:26.813948    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-vk2g6"
May 26 02:36:26.814521 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:26.814497    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-vk2g6"
May 26 02:36:27.041082 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:27.041048    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:27.041253 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:27.041057    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:27.041253 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:27.041172    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:27.041364 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:27.041304    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:27.271696 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:27.271612    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-vk2g6"
May 26 02:36:27.272243 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:27.272186    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-vk2g6"
May 26 02:36:28.274667 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:28.274399    2551 generic.go:358] "Generic (PLEG): container finished" podID="5e26384c-7c3d-4dad-bab9-a74ec1259042" containerID="535db4622fbf8da7fcf2f2cd06cd81e6916109877c78aa565a8f46a52d4187f2" exitCode=0
May 26 02:36:28.274667 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:28.274478    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5x5dt" event={"ID":"5e26384c-7c3d-4dad-bab9-a74ec1259042","Type":"ContainerDied","Data":"535db4622fbf8da7fcf2f2cd06cd81e6916109877c78aa565a8f46a52d4187f2"}
May 26 02:36:28.277664 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:28.277648    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 02:36:28.278060 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:28.278034    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v" event={"ID":"ac847df6-96a0-478b-8c22-796d6809971a","Type":"ContainerStarted","Data":"8b25c6af7f72b3edce5dc306fce0f6f1c1c2e188949b426b60297999b1919d19"}
May 26 02:36:28.278468 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:28.278442    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:28.278572 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:28.278472    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:28.278572 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:28.278485    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:28.278572 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:28.278566    2551 scope.go:117] "RemoveContainer" containerID="fa332139628a699274a6b6f72d0dd52e423fc587037810eef86f7c97b1d218f4"
May 26 02:36:28.293677 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:28.293654    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:28.294486 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:28.294472    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:36:29.040434 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:29.040410    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:29.040568 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:29.040410    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:29.040568 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:29.040538    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:29.040678 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:29.040565    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:29.282809 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:29.282726    2551 generic.go:358] "Generic (PLEG): container finished" podID="5e26384c-7c3d-4dad-bab9-a74ec1259042" containerID="187a57a4ca3607c6f334da174ca3060cf81d0d1197e84c888f770e2487ded1e3" exitCode=0
May 26 02:36:29.282809 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:29.282806    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5x5dt" event={"ID":"5e26384c-7c3d-4dad-bab9-a74ec1259042","Type":"ContainerDied","Data":"187a57a4ca3607c6f334da174ca3060cf81d0d1197e84c888f770e2487ded1e3"}
May 26 02:36:29.287354 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:29.287318    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 02:36:29.288340 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:29.288294    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v" event={"ID":"ac847df6-96a0-478b-8c22-796d6809971a","Type":"ContainerStarted","Data":"14f1df34486fa9f9c789dfa502bb31eddf9358386b25cf25c55c845f1d3f24b6"}
May 26 02:36:29.339153 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:29.339111    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v" podStartSLOduration=8.679681496 podStartE2EDuration="26.33909776s" podCreationTimestamp="2026-05-26 02:36:03 +0000 UTC" firstStartedPulling="2026-05-26 02:36:05.276007904 +0000 UTC m=+2.832796565" lastFinishedPulling="2026-05-26 02:36:22.935424169 +0000 UTC m=+20.492212829" observedRunningTime="2026-05-26 02:36:29.338628691 +0000 UTC m=+26.895417359" watchObservedRunningTime="2026-05-26 02:36:29.33909776 +0000 UTC m=+26.895886428"
May 26 02:36:29.992190 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:29.991975    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29jmf"]
May 26 02:36:29.992324 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:29.992309    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:29.992462 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:29.992440    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:29.994472 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:29.994453    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wzhws"]
May 26 02:36:29.994553 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:29.994526    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:29.994609 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:29.994588    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:30.291614 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:30.291588    2551 generic.go:358] "Generic (PLEG): container finished" podID="5e26384c-7c3d-4dad-bab9-a74ec1259042" containerID="0cf999703c16161573b5d36ec82e0e6824a558c7831dbc32f4ed29bba60b75cb" exitCode=0
May 26 02:36:30.291959 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:30.291670    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5x5dt" event={"ID":"5e26384c-7c3d-4dad-bab9-a74ec1259042","Type":"ContainerDied","Data":"0cf999703c16161573b5d36ec82e0e6824a558c7831dbc32f4ed29bba60b75cb"}
May 26 02:36:32.040624 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:32.040587    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:32.041071 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:32.040587    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:32.041071 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:32.040728    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:32.041071 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:32.040805    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:34.040389 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:34.040354    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:34.041001 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:34.040354    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:34.041001 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:34.040494    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-29jmf" podUID="9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8"
May 26 02:36:34.041001 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:34.040535    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-wzhws" podUID="53dbc3bb-daa3-4d5c-a004-f5046a91b1ae"
May 26 02:36:35.725981 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.725949    2551 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-139-173.ec2.internal" event="NodeReady"
May 26 02:36:35.726544 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.726525    2551 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 26 02:36:35.766630 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.766604    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-r4986"]
May 26 02:36:35.782019 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.781997    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-tpfbl"]
May 26 02:36:35.782150 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.782119    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r4986"
May 26 02:36:35.784170 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.784150    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-77jkf\""
May 26 02:36:35.784370 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.784349    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 26 02:36:35.784461 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.784374    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 26 02:36:35.804147 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.804128    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r4986"]
May 26 02:36:35.804261 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.804151    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tpfbl"]
May 26 02:36:35.804261 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.804236    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:36:35.806415 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.806392    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 26 02:36:35.806506 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.806467    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 26 02:36:35.806506 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.806399    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jxvlq\""
May 26 02:36:35.806603 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.806397    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 26 02:36:35.872096 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.872070    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e61c19-3818-47b3-82b7-bc5b51bb09de-config-volume\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:35.872206 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.872105    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-7gqjr\" (UniqueName: \"kubernetes.io/projected/a7e61c19-3818-47b3-82b7-bc5b51bb09de-kube-api-access-7gqjr\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:35.872206 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.872125    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7e61c19-3818-47b3-82b7-bc5b51bb09de-tmp-dir\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:35.872206 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.872172    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:35.972699 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.972667    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:36:35.972824 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.972714    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e61c19-3818-47b3-82b7-bc5b51bb09de-config-volume\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:35.972824 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.972737    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-7gqjr\" (UniqueName: \"kubernetes.io/projected/a7e61c19-3818-47b3-82b7-bc5b51bb09de-kube-api-access-7gqjr\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:35.972887 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.972847    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7e61c19-3818-47b3-82b7-bc5b51bb09de-tmp-dir\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:35.972952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.972916    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-g9nhx\" (UniqueName: \"kubernetes.io/projected/96779027-bcda-438b-924c-3441dd3c4b9d-kube-api-access-g9nhx\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:36:35.972994 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.972949    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:35.973116 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:35.973099    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 02:36:35.973176 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.973160    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/a7e61c19-3818-47b3-82b7-bc5b51bb09de-tmp-dir\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:35.973209 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:35.973178    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls podName:a7e61c19-3818-47b3-82b7-bc5b51bb09de nodeName:}" failed. No retries permitted until 2026-05-26 02:36:36.473159728 +0000 UTC m=+34.029948380 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls") pod "dns-default-r4986" (UID: "a7e61c19-3818-47b3-82b7-bc5b51bb09de") : secret "dns-default-metrics-tls" not found
May 26 02:36:35.973209 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.973163    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/a7e61c19-3818-47b3-82b7-bc5b51bb09de-config-volume\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:35.982593 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:35.982570    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-7gqjr\" (UniqueName: \"kubernetes.io/projected/a7e61c19-3818-47b3-82b7-bc5b51bb09de-kube-api-access-7gqjr\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:36.040704 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.040675    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:36.040815 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.040685    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:36.044138 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.044107    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 26 02:36:36.044138 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.044125    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 26 02:36:36.044138 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.044128    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-jd67c\""
May 26 02:36:36.044360 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.044176    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 26 02:36:36.044360 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.044107    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kjrvv\""
May 26 02:36:36.073560 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.073542    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-g9nhx\" (UniqueName: \"kubernetes.io/projected/96779027-bcda-438b-924c-3441dd3c4b9d-kube-api-access-g9nhx\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:36:36.073639 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.073587    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:36:36.073730 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:36.073714    2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 02:36:36.073779 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:36.073771    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert podName:96779027-bcda-438b-924c-3441dd3c4b9d nodeName:}" failed. No retries permitted until 2026-05-26 02:36:36.573753045 +0000 UTC m=+34.130541709 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert") pod "ingress-canary-tpfbl" (UID: "96779027-bcda-438b-924c-3441dd3c4b9d") : secret "canary-serving-cert" not found
May 26 02:36:36.082623 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.082600    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-g9nhx\" (UniqueName: \"kubernetes.io/projected/96779027-bcda-438b-924c-3441dd3c4b9d-kube-api-access-g9nhx\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:36:36.305325 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.305297    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5x5dt" event={"ID":"5e26384c-7c3d-4dad-bab9-a74ec1259042","Type":"ContainerStarted","Data":"6fa093de3f9b40ef03348cb7ba9baad398536cf4d3ea29f0c6fabc2bc470f306"}
May 26 02:36:36.475834 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.475761    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:36.476021 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:36.475925    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 02:36:36.476021 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:36.475983    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls podName:a7e61c19-3818-47b3-82b7-bc5b51bb09de nodeName:}" failed. No retries permitted until 2026-05-26 02:36:37.475969594 +0000 UTC m=+35.032758240 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls") pod "dns-default-r4986" (UID: "a7e61c19-3818-47b3-82b7-bc5b51bb09de") : secret "dns-default-metrics-tls" not found
May 26 02:36:36.576819 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.576788    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:36:36.576977 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:36.576920    2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 02:36:36.577018 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:36.576985    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert podName:96779027-bcda-438b-924c-3441dd3c4b9d nodeName:}" failed. No retries permitted until 2026-05-26 02:36:37.576967249 +0000 UTC m=+35.133755914 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert") pod "ingress-canary-tpfbl" (UID: "96779027-bcda-438b-924c-3441dd3c4b9d") : secret "canary-serving-cert" not found
May 26 02:36:36.677153 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.677118    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:36:36.677301 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.677205    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nd54s\" (UniqueName: \"kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s\") pod \"network-check-target-wzhws\" (UID: \"53dbc3bb-daa3-4d5c-a004-f5046a91b1ae\") " pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:36.677301 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:36.677281    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 26 02:36:36.677388 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:36.677349    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs podName:9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8 nodeName:}" failed. No retries permitted until 2026-05-26 02:37:08.677328009 +0000 UTC m=+66.234116672 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs") pod "network-metrics-daemon-29jmf" (UID: "9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8") : secret "metrics-daemon-secret" not found
May 26 02:36:36.679633 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.679614    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nd54s\" (UniqueName: \"kubernetes.io/projected/53dbc3bb-daa3-4d5c-a004-f5046a91b1ae-kube-api-access-nd54s\") pod \"network-check-target-wzhws\" (UID: \"53dbc3bb-daa3-4d5c-a004-f5046a91b1ae\") " pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:36.951115 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:36.951078    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:37.132488 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:37.132272    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-wzhws"]
May 26 02:36:37.136161 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:36:37.136132    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod53dbc3bb_daa3_4d5c_a004_f5046a91b1ae.slice/crio-80c05ecb2389dfefb8e2325aec9c8681d3449e2f056864adb5383de408889af9 WatchSource:0}: Error finding container 80c05ecb2389dfefb8e2325aec9c8681d3449e2f056864adb5383de408889af9: Status 404 returned error can't find the container with id 80c05ecb2389dfefb8e2325aec9c8681d3449e2f056864adb5383de408889af9
May 26 02:36:37.309525 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:37.309497    2551 generic.go:358] "Generic (PLEG): container finished" podID="5e26384c-7c3d-4dad-bab9-a74ec1259042" containerID="6fa093de3f9b40ef03348cb7ba9baad398536cf4d3ea29f0c6fabc2bc470f306" exitCode=0
May 26 02:36:37.309693 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:37.309584    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5x5dt" event={"ID":"5e26384c-7c3d-4dad-bab9-a74ec1259042","Type":"ContainerDied","Data":"6fa093de3f9b40ef03348cb7ba9baad398536cf4d3ea29f0c6fabc2bc470f306"}
May 26 02:36:37.310595 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:37.310577    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wzhws" event={"ID":"53dbc3bb-daa3-4d5c-a004-f5046a91b1ae","Type":"ContainerStarted","Data":"80c05ecb2389dfefb8e2325aec9c8681d3449e2f056864adb5383de408889af9"}
May 26 02:36:37.482915 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:37.482873    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:37.483027 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:37.482999    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 02:36:37.483077 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:37.483071    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls podName:a7e61c19-3818-47b3-82b7-bc5b51bb09de nodeName:}" failed. No retries permitted until 2026-05-26 02:36:39.483051451 +0000 UTC m=+37.039840106 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls") pod "dns-default-r4986" (UID: "a7e61c19-3818-47b3-82b7-bc5b51bb09de") : secret "dns-default-metrics-tls" not found
May 26 02:36:37.584035 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:37.583960    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:36:37.584202 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:37.584102    2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 02:36:37.584202 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:37.584182    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert podName:96779027-bcda-438b-924c-3441dd3c4b9d nodeName:}" failed. No retries permitted until 2026-05-26 02:36:39.584159498 +0000 UTC m=+37.140948151 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert") pod "ingress-canary-tpfbl" (UID: "96779027-bcda-438b-924c-3441dd3c4b9d") : secret "canary-serving-cert" not found
May 26 02:36:38.316014 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:38.315983    2551 generic.go:358] "Generic (PLEG): container finished" podID="5e26384c-7c3d-4dad-bab9-a74ec1259042" containerID="c57a994719a497b1e086f6b889f1b6da27748ffbc4c4f3baecb669cf9ada751b" exitCode=0
May 26 02:36:38.316524 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:38.316036    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5x5dt" event={"ID":"5e26384c-7c3d-4dad-bab9-a74ec1259042","Type":"ContainerDied","Data":"c57a994719a497b1e086f6b889f1b6da27748ffbc4c4f3baecb669cf9ada751b"}
May 26 02:36:39.322249 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:39.321994    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-5x5dt" event={"ID":"5e26384c-7c3d-4dad-bab9-a74ec1259042","Type":"ContainerStarted","Data":"7ee8e5aa8ba0c4bbf4711262418ade781c6a7f4229b339b1e72eb3b856d5b692"}
May 26 02:36:39.344778 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:39.344442    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-5x5dt" podStartSLOduration=5.579701454 podStartE2EDuration="36.34442376s" podCreationTimestamp="2026-05-26 02:36:03 +0000 UTC" firstStartedPulling="2026-05-26 02:36:05.281348887 +0000 UTC m=+2.838137539" lastFinishedPulling="2026-05-26 02:36:36.046071197 +0000 UTC m=+33.602859845" observedRunningTime="2026-05-26 02:36:39.342938428 +0000 UTC m=+36.899727098" watchObservedRunningTime="2026-05-26 02:36:39.34442376 +0000 UTC m=+36.901212430"
May 26 02:36:39.498584 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:39.498549    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:39.498770 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:39.498734    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 02:36:39.498836 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:39.498808    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls podName:a7e61c19-3818-47b3-82b7-bc5b51bb09de nodeName:}" failed. No retries permitted until 2026-05-26 02:36:43.498786361 +0000 UTC m=+41.055575028 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls") pod "dns-default-r4986" (UID: "a7e61c19-3818-47b3-82b7-bc5b51bb09de") : secret "dns-default-metrics-tls" not found
May 26 02:36:39.600024 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:39.599933    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:36:39.600177 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:39.600089    2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 02:36:39.600177 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:39.600154    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert podName:96779027-bcda-438b-924c-3441dd3c4b9d nodeName:}" failed. No retries permitted until 2026-05-26 02:36:43.600138656 +0000 UTC m=+41.156927302 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert") pod "ingress-canary-tpfbl" (UID: "96779027-bcda-438b-924c-3441dd3c4b9d") : secret "canary-serving-cert" not found
May 26 02:36:41.326220 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:41.326178    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-wzhws" event={"ID":"53dbc3bb-daa3-4d5c-a004-f5046a91b1ae","Type":"ContainerStarted","Data":"ebbdc19a23c0ebcdb614b95dd970f56050c286a3c967adc9110ac4e8ec5c91ba"}
May 26 02:36:41.326714 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:41.326333    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:36:41.341398 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:41.341353    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-wzhws" podStartSLOduration=35.144679143 podStartE2EDuration="38.341331057s" podCreationTimestamp="2026-05-26 02:36:03 +0000 UTC" firstStartedPulling="2026-05-26 02:36:37.13805003 +0000 UTC m=+34.694838690" lastFinishedPulling="2026-05-26 02:36:40.334701953 +0000 UTC m=+37.891490604" observedRunningTime="2026-05-26 02:36:41.341122046 +0000 UTC m=+38.897910717" watchObservedRunningTime="2026-05-26 02:36:41.341331057 +0000 UTC m=+38.898119727"
May 26 02:36:43.524097 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:43.524059    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:43.524450 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:43.524207    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 02:36:43.524450 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:43.524272    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls podName:a7e61c19-3818-47b3-82b7-bc5b51bb09de nodeName:}" failed. No retries permitted until 2026-05-26 02:36:51.524256289 +0000 UTC m=+49.081044935 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls") pod "dns-default-r4986" (UID: "a7e61c19-3818-47b3-82b7-bc5b51bb09de") : secret "dns-default-metrics-tls" not found
May 26 02:36:43.625212 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:43.625179    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:36:43.625361 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:43.625312    2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 02:36:43.625410 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:43.625373    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert podName:96779027-bcda-438b-924c-3441dd3c4b9d nodeName:}" failed. No retries permitted until 2026-05-26 02:36:51.625358781 +0000 UTC m=+49.182147434 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert") pod "ingress-canary-tpfbl" (UID: "96779027-bcda-438b-924c-3441dd3c4b9d") : secret "canary-serving-cert" not found
May 26 02:36:51.577064 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:51.577023    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:36:51.577518 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:51.577151    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 02:36:51.577518 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:51.577200    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls podName:a7e61c19-3818-47b3-82b7-bc5b51bb09de nodeName:}" failed. No retries permitted until 2026-05-26 02:37:07.577186804 +0000 UTC m=+65.133975452 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls") pod "dns-default-r4986" (UID: "a7e61c19-3818-47b3-82b7-bc5b51bb09de") : secret "dns-default-metrics-tls" not found
May 26 02:36:51.677787 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:36:51.677759    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:36:51.677919 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:51.677878    2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 02:36:51.677963 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:36:51.677950    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert podName:96779027-bcda-438b-924c-3441dd3c4b9d nodeName:}" failed. No retries permitted until 2026-05-26 02:37:07.677935581 +0000 UTC m=+65.234724233 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert") pod "ingress-canary-tpfbl" (UID: "96779027-bcda-438b-924c-3441dd3c4b9d") : secret "canary-serving-cert" not found
May 26 02:37:00.301963 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:00.301935    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-t8q5v"
May 26 02:37:07.584714 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:07.584671    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:37:07.585187 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:07.584830    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 02:37:07.585187 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:07.584916    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls podName:a7e61c19-3818-47b3-82b7-bc5b51bb09de nodeName:}" failed. No retries permitted until 2026-05-26 02:37:39.584880993 +0000 UTC m=+97.141669658 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls") pod "dns-default-r4986" (UID: "a7e61c19-3818-47b3-82b7-bc5b51bb09de") : secret "dns-default-metrics-tls" not found
May 26 02:37:07.685811 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:07.685774    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:37:07.686001 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:07.685935    2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 02:37:07.686044 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:07.686018    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert podName:96779027-bcda-438b-924c-3441dd3c4b9d nodeName:}" failed. No retries permitted until 2026-05-26 02:37:39.686002577 +0000 UTC m=+97.242791223 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert") pod "ingress-canary-tpfbl" (UID: "96779027-bcda-438b-924c-3441dd3c4b9d") : secret "canary-serving-cert" not found
May 26 02:37:08.693682 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:08.693643    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:37:08.694087 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:08.693768    2551 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 26 02:37:08.694087 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:08.693823    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs podName:9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8 nodeName:}" failed. No retries permitted until 2026-05-26 02:38:12.693807518 +0000 UTC m=+130.250596165 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs") pod "network-metrics-daemon-29jmf" (UID: "9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8") : secret "metrics-daemon-secret" not found
May 26 02:37:12.330437 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:12.330409    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-wzhws"
May 26 02:37:39.602234 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:39.602190    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:37:39.602649 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:39.602338    2551 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 26 02:37:39.602649 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:39.602414    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls podName:a7e61c19-3818-47b3-82b7-bc5b51bb09de nodeName:}" failed. No retries permitted until 2026-05-26 02:38:43.602398352 +0000 UTC m=+161.159186998 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls") pod "dns-default-r4986" (UID: "a7e61c19-3818-47b3-82b7-bc5b51bb09de") : secret "dns-default-metrics-tls" not found
May 26 02:37:39.703417 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:39.703382    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:37:39.703569 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:39.703521    2551 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 26 02:37:39.703606 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:39.703586    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert podName:96779027-bcda-438b-924c-3441dd3c4b9d nodeName:}" failed. No retries permitted until 2026-05-26 02:38:43.703569972 +0000 UTC m=+161.260358620 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert") pod "ingress-canary-tpfbl" (UID: "96779027-bcda-438b-924c-3441dd3c4b9d") : secret "canary-serving-cert" not found
May 26 02:37:49.072813 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.072778    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-g2gst"]
May 26 02:37:49.075585 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.075564    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-g2gst"
May 26 02:37:49.075694 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.075605    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-5pgsl"]
May 26 02:37:49.077830 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.077814    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"kube-root-ca.crt\""
May 26 02:37:49.078266 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.078248    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.078651 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.078635    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-storage-operator\"/\"openshift-service-ca.crt\""
May 26 02:37:49.078726 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.078652    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-storage-operator\"/\"volume-data-source-validator-dockercfg-hqtgg\""
May 26 02:37:49.080063 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.080045    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-p9x4q"]
May 26 02:37:49.080486 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.080470    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
May 26 02:37:49.081059 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.081042    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
May 26 02:37:49.081163 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.081063    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-7v6v9\""
May 26 02:37:49.081163 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.081068    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
May 26 02:37:49.081373 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.081342    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
May 26 02:37:49.082794 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.082771    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-g2gst"]
May 26 02:37:49.082932 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.082914    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.090759 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.090729    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
May 26 02:37:49.091128 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.091094    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 26 02:37:49.091320 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.091304    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 26 02:37:49.091620 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.091599    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"operator-dockercfg-ntk49\""
May 26 02:37:49.091880 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.091792    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"service-ca-bundle\""
May 26 02:37:49.091880 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.091846    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"openshift-insights-serving-cert\""
May 26 02:37:49.092942 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.092917    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-5pgsl"]
May 26 02:37:49.094661 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.094633    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-p9x4q"]
May 26 02:37:49.096056 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.096036    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"trusted-ca-bundle\""
May 26 02:37:49.165150 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.165116    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb87f3ac-7bba-4002-8d34-fadbd525b32a-config\") pod \"console-operator-76dff48d59-5pgsl\" (UID: \"fb87f3ac-7bba-4002-8d34-fadbd525b32a\") " pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.165150 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.165148    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1511080f-2218-4a2b-8395-1ae813c119b2-serving-cert\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.165361 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.165189    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb87f3ac-7bba-4002-8d34-fadbd525b32a-serving-cert\") pod \"console-operator-76dff48d59-5pgsl\" (UID: \"fb87f3ac-7bba-4002-8d34-fadbd525b32a\") " pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.165361 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.165217    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-67ztr\" (UniqueName: \"kubernetes.io/projected/974b2dec-e8a5-4ac2-af9d-46e5acdfc7f4-kube-api-access-67ztr\") pod \"volume-data-source-validator-5cf8c89d64-g2gst\" (UID: \"974b2dec-e8a5-4ac2-af9d-46e5acdfc7f4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-g2gst"
May 26 02:37:49.165361 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.165234    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1511080f-2218-4a2b-8395-1ae813c119b2-service-ca-bundle\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.165361 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.165296    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb87f3ac-7bba-4002-8d34-fadbd525b32a-trusted-ca\") pod \"console-operator-76dff48d59-5pgsl\" (UID: \"fb87f3ac-7bba-4002-8d34-fadbd525b32a\") " pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.165361 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.165326    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1511080f-2218-4a2b-8395-1ae813c119b2-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.165361 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.165347    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1511080f-2218-4a2b-8395-1ae813c119b2-tmp\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.165537 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.165408    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1511080f-2218-4a2b-8395-1ae813c119b2-snapshots\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.165537 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.165424    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8mw2c\" (UniqueName: \"kubernetes.io/projected/1511080f-2218-4a2b-8395-1ae813c119b2-kube-api-access-8mw2c\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.165537 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.165446    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qpgxn\" (UniqueName: \"kubernetes.io/projected/fb87f3ac-7bba-4002-8d34-fadbd525b32a-kube-api-access-qpgxn\") pod \"console-operator-76dff48d59-5pgsl\" (UID: \"fb87f3ac-7bba-4002-8d34-fadbd525b32a\") " pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.176648 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.176622    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-56c484c8cc-pr8sr"]
May 26 02:37:49.179533 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.179519    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.181793 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.181771    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 26 02:37:49.182292 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.182277    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 26 02:37:49.182386 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.182285    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bpf8g\""
May 26 02:37:49.182386 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.182296    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 26 02:37:49.189042 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.189025    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 26 02:37:49.193770 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.193752    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56c484c8cc-pr8sr"]
May 26 02:37:49.266459 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266420    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a121687e-575e-450a-b679-7ff6c044fc48-registry-certificates\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.266459 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266463    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qpgxn\" (UniqueName: \"kubernetes.io/projected/fb87f3ac-7bba-4002-8d34-fadbd525b32a-kube-api-access-qpgxn\") pod \"console-operator-76dff48d59-5pgsl\" (UID: \"fb87f3ac-7bba-4002-8d34-fadbd525b32a\") " pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.266709 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266496    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb87f3ac-7bba-4002-8d34-fadbd525b32a-config\") pod \"console-operator-76dff48d59-5pgsl\" (UID: \"fb87f3ac-7bba-4002-8d34-fadbd525b32a\") " pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.266709 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266515    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1511080f-2218-4a2b-8395-1ae813c119b2-serving-cert\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.266709 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266533    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-bound-sa-token\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.266709 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266550    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a121687e-575e-450a-b679-7ff6c044fc48-ca-trust-extracted\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.266709 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266586    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb87f3ac-7bba-4002-8d34-fadbd525b32a-serving-cert\") pod \"console-operator-76dff48d59-5pgsl\" (UID: \"fb87f3ac-7bba-4002-8d34-fadbd525b32a\") " pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.266709 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266648    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-67ztr\" (UniqueName: \"kubernetes.io/projected/974b2dec-e8a5-4ac2-af9d-46e5acdfc7f4-kube-api-access-67ztr\") pod \"volume-data-source-validator-5cf8c89d64-g2gst\" (UID: \"974b2dec-e8a5-4ac2-af9d-46e5acdfc7f4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-g2gst"
May 26 02:37:49.267038 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266784    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1511080f-2218-4a2b-8395-1ae813c119b2-service-ca-bundle\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.267038 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266828    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a121687e-575e-450a-b679-7ff6c044fc48-trusted-ca\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.267038 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266855    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a121687e-575e-450a-b679-7ff6c044fc48-installation-pull-secrets\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.267038 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266891    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb87f3ac-7bba-4002-8d34-fadbd525b32a-trusted-ca\") pod \"console-operator-76dff48d59-5pgsl\" (UID: \"fb87f3ac-7bba-4002-8d34-fadbd525b32a\") " pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.267038 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266937    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1511080f-2218-4a2b-8395-1ae813c119b2-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.267038 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.266967    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a121687e-575e-450a-b679-7ff6c044fc48-image-registry-private-configuration\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.267038 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.267000    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1511080f-2218-4a2b-8395-1ae813c119b2-tmp\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.267038 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.267026    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-csh5x\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-kube-api-access-csh5x\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.267418 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.267054    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.267418 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.267103    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1511080f-2218-4a2b-8395-1ae813c119b2-snapshots\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.267418 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.267127    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8mw2c\" (UniqueName: \"kubernetes.io/projected/1511080f-2218-4a2b-8395-1ae813c119b2-kube-api-access-8mw2c\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.267565 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.267476    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1511080f-2218-4a2b-8395-1ae813c119b2-service-ca-bundle\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.267565 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.267494    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/fb87f3ac-7bba-4002-8d34-fadbd525b32a-config\") pod \"console-operator-76dff48d59-5pgsl\" (UID: \"fb87f3ac-7bba-4002-8d34-fadbd525b32a\") " pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.267675 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.267656    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/1511080f-2218-4a2b-8395-1ae813c119b2-tmp\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.268010 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.267987    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"snapshots\" (UniqueName: \"kubernetes.io/empty-dir/1511080f-2218-4a2b-8395-1ae813c119b2-snapshots\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.268097 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.268082    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/1511080f-2218-4a2b-8395-1ae813c119b2-trusted-ca-bundle\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.268294 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.268269    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fb87f3ac-7bba-4002-8d34-fadbd525b32a-trusted-ca\") pod \"console-operator-76dff48d59-5pgsl\" (UID: \"fb87f3ac-7bba-4002-8d34-fadbd525b32a\") " pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.270607 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.270588    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/1511080f-2218-4a2b-8395-1ae813c119b2-serving-cert\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.270678 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.270591    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/fb87f3ac-7bba-4002-8d34-fadbd525b32a-serving-cert\") pod \"console-operator-76dff48d59-5pgsl\" (UID: \"fb87f3ac-7bba-4002-8d34-fadbd525b32a\") " pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.274445 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.274421    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8mw2c\" (UniqueName: \"kubernetes.io/projected/1511080f-2218-4a2b-8395-1ae813c119b2-kube-api-access-8mw2c\") pod \"insights-operator-78bdfb888c-p9x4q\" (UID: \"1511080f-2218-4a2b-8395-1ae813c119b2\") " pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.274750 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.274712    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qpgxn\" (UniqueName: \"kubernetes.io/projected/fb87f3ac-7bba-4002-8d34-fadbd525b32a-kube-api-access-qpgxn\") pod \"console-operator-76dff48d59-5pgsl\" (UID: \"fb87f3ac-7bba-4002-8d34-fadbd525b32a\") " pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.274837 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.274817    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-67ztr\" (UniqueName: \"kubernetes.io/projected/974b2dec-e8a5-4ac2-af9d-46e5acdfc7f4-kube-api-access-67ztr\") pod \"volume-data-source-validator-5cf8c89d64-g2gst\" (UID: \"974b2dec-e8a5-4ac2-af9d-46e5acdfc7f4\") " pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-g2gst"
May 26 02:37:49.368198 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.368106    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-bound-sa-token\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.368198 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.368147    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a121687e-575e-450a-b679-7ff6c044fc48-ca-trust-extracted\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.368396 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.368305    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a121687e-575e-450a-b679-7ff6c044fc48-trusted-ca\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.368396 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.368343    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a121687e-575e-450a-b679-7ff6c044fc48-installation-pull-secrets\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.368508 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.368451    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a121687e-575e-450a-b679-7ff6c044fc48-ca-trust-extracted\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.368508 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.368469    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a121687e-575e-450a-b679-7ff6c044fc48-image-registry-private-configuration\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.368610 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.368508    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-csh5x\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-kube-api-access-csh5x\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.368610 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.368540    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.368610 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.368587    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a121687e-575e-450a-b679-7ff6c044fc48-registry-certificates\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.368764 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:49.368705    2551 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 02:37:49.368764 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:49.368724    2551 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c484c8cc-pr8sr: secret "image-registry-tls" not found
May 26 02:37:49.368871 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:49.368796    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls podName:a121687e-575e-450a-b679-7ff6c044fc48 nodeName:}" failed. No retries permitted until 2026-05-26 02:37:49.86877558 +0000 UTC m=+107.425564248 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls") pod "image-registry-56c484c8cc-pr8sr" (UID: "a121687e-575e-450a-b679-7ff6c044fc48") : secret "image-registry-tls" not found
May 26 02:37:49.369122 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.369102    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a121687e-575e-450a-b679-7ff6c044fc48-registry-certificates\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.369279 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.369258    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a121687e-575e-450a-b679-7ff6c044fc48-trusted-ca\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.370705 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.370683    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a121687e-575e-450a-b679-7ff6c044fc48-installation-pull-secrets\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.370787 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.370724    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a121687e-575e-450a-b679-7ff6c044fc48-image-registry-private-configuration\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.376553 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.376528    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-bound-sa-token\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.376553 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.376540    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-csh5x\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-kube-api-access-csh5x\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.390565 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.390542    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-g2gst"
May 26 02:37:49.397229 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.397206    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:49.402804 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.402786    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-operator-78bdfb888c-p9x4q"
May 26 02:37:49.519786 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.519745    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-g2gst"]
May 26 02:37:49.523951 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:37:49.523925    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod974b2dec_e8a5_4ac2_af9d_46e5acdfc7f4.slice/crio-1f6ed6f952944c7c8ded753ad7129d0d1285ab13dfa8f6741c07f9c49bdf0ac1 WatchSource:0}: Error finding container 1f6ed6f952944c7c8ded753ad7129d0d1285ab13dfa8f6741c07f9c49bdf0ac1: Status 404 returned error can't find the container with id 1f6ed6f952944c7c8ded753ad7129d0d1285ab13dfa8f6741c07f9c49bdf0ac1
May 26 02:37:49.544852 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.544827    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-5pgsl"]
May 26 02:37:49.549948 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:37:49.549888    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfb87f3ac_7bba_4002_8d34_fadbd525b32a.slice/crio-cae75f6cba46d5dea472277ee285fa10023a5d0cd16f43e8d205fd0f8e0837e3 WatchSource:0}: Error finding container cae75f6cba46d5dea472277ee285fa10023a5d0cd16f43e8d205fd0f8e0837e3: Status 404 returned error can't find the container with id cae75f6cba46d5dea472277ee285fa10023a5d0cd16f43e8d205fd0f8e0837e3
May 26 02:37:49.556768 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.556717    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-operator-78bdfb888c-p9x4q"]
May 26 02:37:49.559871 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:37:49.559849    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1511080f_2218_4a2b_8395_1ae813c119b2.slice/crio-a27cc407ed5e48ffd7c5f14ab80dd4f1f30f0b78f41539a96d2b2854966f87a4 WatchSource:0}: Error finding container a27cc407ed5e48ffd7c5f14ab80dd4f1f30f0b78f41539a96d2b2854966f87a4: Status 404 returned error can't find the container with id a27cc407ed5e48ffd7c5f14ab80dd4f1f30f0b78f41539a96d2b2854966f87a4
May 26 02:37:49.871500 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.871461    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:49.871663 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:49.871583    2551 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 02:37:49.871663 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:49.871598    2551 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c484c8cc-pr8sr: secret "image-registry-tls" not found
May 26 02:37:49.871663 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:49.871653    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls podName:a121687e-575e-450a-b679-7ff6c044fc48 nodeName:}" failed. No retries permitted until 2026-05-26 02:37:50.871637065 +0000 UTC m=+108.428425718 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls") pod "image-registry-56c484c8cc-pr8sr" (UID: "a121687e-575e-450a-b679-7ff6c044fc48") : secret "image-registry-tls" not found
May 26 02:37:49.937711 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.937678    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"]
May 26 02:37:49.942017 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.941994    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"
May 26 02:37:49.944240 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.944209    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
May 26 02:37:49.944372 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.944351    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
May 26 02:37:49.944456 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.944366    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-vxhws\""
May 26 02:37:49.944456 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.944417    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
May 26 02:37:49.944456 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.944437    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
May 26 02:37:49.953445 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:49.951925    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"]
May 26 02:37:50.073174 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.073130    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/355b4827-cbde-4c11-813c-08ed3b73ad7d-config\") pod \"kube-storage-version-migrator-operator-565644f49-xs5pj\" (UID: \"355b4827-cbde-4c11-813c-08ed3b73ad7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"
May 26 02:37:50.073586 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.073205    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/355b4827-cbde-4c11-813c-08ed3b73ad7d-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-xs5pj\" (UID: \"355b4827-cbde-4c11-813c-08ed3b73ad7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"
May 26 02:37:50.073586 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.073232    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ctcph\" (UniqueName: \"kubernetes.io/projected/355b4827-cbde-4c11-813c-08ed3b73ad7d-kube-api-access-ctcph\") pod \"kube-storage-version-migrator-operator-565644f49-xs5pj\" (UID: \"355b4827-cbde-4c11-813c-08ed3b73ad7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"
May 26 02:37:50.173955 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.173843    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/355b4827-cbde-4c11-813c-08ed3b73ad7d-config\") pod \"kube-storage-version-migrator-operator-565644f49-xs5pj\" (UID: \"355b4827-cbde-4c11-813c-08ed3b73ad7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"
May 26 02:37:50.173955 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.173948    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/355b4827-cbde-4c11-813c-08ed3b73ad7d-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-xs5pj\" (UID: \"355b4827-cbde-4c11-813c-08ed3b73ad7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"
May 26 02:37:50.174161 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.173977    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ctcph\" (UniqueName: \"kubernetes.io/projected/355b4827-cbde-4c11-813c-08ed3b73ad7d-kube-api-access-ctcph\") pod \"kube-storage-version-migrator-operator-565644f49-xs5pj\" (UID: \"355b4827-cbde-4c11-813c-08ed3b73ad7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"
May 26 02:37:50.174548 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.174493    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/355b4827-cbde-4c11-813c-08ed3b73ad7d-config\") pod \"kube-storage-version-migrator-operator-565644f49-xs5pj\" (UID: \"355b4827-cbde-4c11-813c-08ed3b73ad7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"
May 26 02:37:50.176964 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.176937    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/355b4827-cbde-4c11-813c-08ed3b73ad7d-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-xs5pj\" (UID: \"355b4827-cbde-4c11-813c-08ed3b73ad7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"
May 26 02:37:50.182813 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.182783    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ctcph\" (UniqueName: \"kubernetes.io/projected/355b4827-cbde-4c11-813c-08ed3b73ad7d-kube-api-access-ctcph\") pod \"kube-storage-version-migrator-operator-565644f49-xs5pj\" (UID: \"355b4827-cbde-4c11-813c-08ed3b73ad7d\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"
May 26 02:37:50.252401 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.252110    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"
May 26 02:37:50.419238 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.419204    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj"]
May 26 02:37:50.423646 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:37:50.423608    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod355b4827_cbde_4c11_813c_08ed3b73ad7d.slice/crio-3248ec0704c37a5eb703f668b75bbb45275a94ec34aa4ed85c2783cfb638a283 WatchSource:0}: Error finding container 3248ec0704c37a5eb703f668b75bbb45275a94ec34aa4ed85c2783cfb638a283: Status 404 returned error can't find the container with id 3248ec0704c37a5eb703f668b75bbb45275a94ec34aa4ed85c2783cfb638a283
May 26 02:37:50.455774 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.455730    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-g2gst" event={"ID":"974b2dec-e8a5-4ac2-af9d-46e5acdfc7f4","Type":"ContainerStarted","Data":"1f6ed6f952944c7c8ded753ad7129d0d1285ab13dfa8f6741c07f9c49bdf0ac1"}
May 26 02:37:50.457725 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.457685    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj" event={"ID":"355b4827-cbde-4c11-813c-08ed3b73ad7d","Type":"ContainerStarted","Data":"3248ec0704c37a5eb703f668b75bbb45275a94ec34aa4ed85c2783cfb638a283"}
May 26 02:37:50.458842 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.458814    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl" event={"ID":"fb87f3ac-7bba-4002-8d34-fadbd525b32a","Type":"ContainerStarted","Data":"cae75f6cba46d5dea472277ee285fa10023a5d0cd16f43e8d205fd0f8e0837e3"}
May 26 02:37:50.460515 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.460487    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-p9x4q" event={"ID":"1511080f-2218-4a2b-8395-1ae813c119b2","Type":"ContainerStarted","Data":"a27cc407ed5e48ffd7c5f14ab80dd4f1f30f0b78f41539a96d2b2854966f87a4"}
May 26 02:37:50.881265 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:50.881222    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:50.881463 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:50.881399    2551 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 02:37:50.881463 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:50.881427    2551 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c484c8cc-pr8sr: secret "image-registry-tls" not found
May 26 02:37:50.881578 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:50.881498    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls podName:a121687e-575e-450a-b679-7ff6c044fc48 nodeName:}" failed. No retries permitted until 2026-05-26 02:37:52.881476362 +0000 UTC m=+110.438265016 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls") pod "image-registry-56c484c8cc-pr8sr" (UID: "a121687e-575e-450a-b679-7ff6c044fc48") : secret "image-registry-tls" not found
May 26 02:37:51.936228 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:51.936195    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"]
May 26 02:37:51.940531 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:51.940512    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"
May 26 02:37:51.942738 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:51.942713    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
May 26 02:37:51.942881 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:51.942861    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rkqwh\""
May 26 02:37:51.942977 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:51.942956    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
May 26 02:37:51.943729 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:51.943711    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
May 26 02:37:51.943818 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:51.943772    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
May 26 02:37:51.948750 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:51.948446    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"]
May 26 02:37:52.093545 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:52.093507    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/49675dc6-0f22-46e0-a1cd-5408f5598780-serving-cert\") pod \"service-ca-operator-57b5b68b8-wptxx\" (UID: \"49675dc6-0f22-46e0-a1cd-5408f5598780\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"
May 26 02:37:52.093710 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:52.093569    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ssg2r\" (UniqueName: \"kubernetes.io/projected/49675dc6-0f22-46e0-a1cd-5408f5598780-kube-api-access-ssg2r\") pod \"service-ca-operator-57b5b68b8-wptxx\" (UID: \"49675dc6-0f22-46e0-a1cd-5408f5598780\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"
May 26 02:37:52.093710 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:52.093640    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/49675dc6-0f22-46e0-a1cd-5408f5598780-config\") pod \"service-ca-operator-57b5b68b8-wptxx\" (UID: \"49675dc6-0f22-46e0-a1cd-5408f5598780\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"
May 26 02:37:52.194340 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:52.194259    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/49675dc6-0f22-46e0-a1cd-5408f5598780-config\") pod \"service-ca-operator-57b5b68b8-wptxx\" (UID: \"49675dc6-0f22-46e0-a1cd-5408f5598780\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"
May 26 02:37:52.194497 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:52.194377    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/49675dc6-0f22-46e0-a1cd-5408f5598780-serving-cert\") pod \"service-ca-operator-57b5b68b8-wptxx\" (UID: \"49675dc6-0f22-46e0-a1cd-5408f5598780\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"
May 26 02:37:52.194497 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:52.194428    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ssg2r\" (UniqueName: \"kubernetes.io/projected/49675dc6-0f22-46e0-a1cd-5408f5598780-kube-api-access-ssg2r\") pod \"service-ca-operator-57b5b68b8-wptxx\" (UID: \"49675dc6-0f22-46e0-a1cd-5408f5598780\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"
May 26 02:37:52.194924 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:52.194881    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/49675dc6-0f22-46e0-a1cd-5408f5598780-config\") pod \"service-ca-operator-57b5b68b8-wptxx\" (UID: \"49675dc6-0f22-46e0-a1cd-5408f5598780\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"
May 26 02:37:52.196790 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:52.196770    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/49675dc6-0f22-46e0-a1cd-5408f5598780-serving-cert\") pod \"service-ca-operator-57b5b68b8-wptxx\" (UID: \"49675dc6-0f22-46e0-a1cd-5408f5598780\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"
May 26 02:37:52.202836 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:52.202817    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ssg2r\" (UniqueName: \"kubernetes.io/projected/49675dc6-0f22-46e0-a1cd-5408f5598780-kube-api-access-ssg2r\") pod \"service-ca-operator-57b5b68b8-wptxx\" (UID: \"49675dc6-0f22-46e0-a1cd-5408f5598780\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"
May 26 02:37:52.251825 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:52.251782    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"
May 26 02:37:52.739711 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:52.738958    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx"]
May 26 02:37:52.788839 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:37:52.788803    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod49675dc6_0f22_46e0_a1cd_5408f5598780.slice/crio-4c5fb960d87e7fec74b9dcbd0b2d34b4aafd22cadc7d6766ad4a09b84ef0870a WatchSource:0}: Error finding container 4c5fb960d87e7fec74b9dcbd0b2d34b4aafd22cadc7d6766ad4a09b84ef0870a: Status 404 returned error can't find the container with id 4c5fb960d87e7fec74b9dcbd0b2d34b4aafd22cadc7d6766ad4a09b84ef0870a
May 26 02:37:52.899708 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:52.899613    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:52.899855 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:52.899764    2551 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 02:37:52.899855 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:52.899787    2551 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c484c8cc-pr8sr: secret "image-registry-tls" not found
May 26 02:37:52.899855 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:52.899849    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls podName:a121687e-575e-450a-b679-7ff6c044fc48 nodeName:}" failed. No retries permitted until 2026-05-26 02:37:56.899832557 +0000 UTC m=+114.456621204 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls") pod "image-registry-56c484c8cc-pr8sr" (UID: "a121687e-575e-450a-b679-7ff6c044fc48") : secret "image-registry-tls" not found
May 26 02:37:53.470365 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:53.470025    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj" event={"ID":"355b4827-cbde-4c11-813c-08ed3b73ad7d","Type":"ContainerStarted","Data":"c88d6c54a5fc2e00b036d9e430c0149f598dc68e6aa3af21385834f21fea03fa"}
May 26 02:37:53.471986 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:53.471705    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/0.log"
May 26 02:37:53.471986 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:53.471747    2551 generic.go:358] "Generic (PLEG): container finished" podID="fb87f3ac-7bba-4002-8d34-fadbd525b32a" containerID="db9ed92574de126ff06d8cecdbb8df045123b5dca032c38df3a49bc4c06d5be2" exitCode=255
May 26 02:37:53.471986 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:53.471917    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl" event={"ID":"fb87f3ac-7bba-4002-8d34-fadbd525b32a","Type":"ContainerDied","Data":"db9ed92574de126ff06d8cecdbb8df045123b5dca032c38df3a49bc4c06d5be2"}
May 26 02:37:53.472216 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:53.472133    2551 scope.go:117] "RemoveContainer" containerID="db9ed92574de126ff06d8cecdbb8df045123b5dca032c38df3a49bc4c06d5be2"
May 26 02:37:53.474045 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:53.474020    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-p9x4q" event={"ID":"1511080f-2218-4a2b-8395-1ae813c119b2","Type":"ContainerStarted","Data":"e201c7740d7c4d8750f1005ba291f474b695b8cd89fea502494aa6a109793afa"}
May 26 02:37:53.476116 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:53.476079    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-g2gst" event={"ID":"974b2dec-e8a5-4ac2-af9d-46e5acdfc7f4","Type":"ContainerStarted","Data":"ca2d6f518d4b4bc1c082e1c2f42e046e77ae0f46287bab37eba8406cae766aa1"}
May 26 02:37:53.477021 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:53.476998    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx" event={"ID":"49675dc6-0f22-46e0-a1cd-5408f5598780","Type":"ContainerStarted","Data":"4c5fb960d87e7fec74b9dcbd0b2d34b4aafd22cadc7d6766ad4a09b84ef0870a"}
May 26 02:37:53.503314 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:53.503261    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj" podStartSLOduration=2.278843773 podStartE2EDuration="4.50324235s" podCreationTimestamp="2026-05-26 02:37:49 +0000 UTC" firstStartedPulling="2026-05-26 02:37:50.426432533 +0000 UTC m=+107.983221186" lastFinishedPulling="2026-05-26 02:37:52.650831102 +0000 UTC m=+110.207619763" observedRunningTime="2026-05-26 02:37:53.486284245 +0000 UTC m=+111.043072915" watchObservedRunningTime="2026-05-26 02:37:53.50324235 +0000 UTC m=+111.060031019"
May 26 02:37:53.503848 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:53.503797    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-operator-78bdfb888c-p9x4q" podStartSLOduration=1.4738096889999999 podStartE2EDuration="4.503786711s" podCreationTimestamp="2026-05-26 02:37:49 +0000 UTC" firstStartedPulling="2026-05-26 02:37:49.561515908 +0000 UTC m=+107.118304555" lastFinishedPulling="2026-05-26 02:37:52.591492919 +0000 UTC m=+110.148281577" observedRunningTime="2026-05-26 02:37:53.502666008 +0000 UTC m=+111.059454677" watchObservedRunningTime="2026-05-26 02:37:53.503786711 +0000 UTC m=+111.060575383"
May 26 02:37:53.533942 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:53.533752    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-storage-operator/volume-data-source-validator-5cf8c89d64-g2gst" podStartSLOduration=1.468406132 podStartE2EDuration="4.533731168s" podCreationTimestamp="2026-05-26 02:37:49 +0000 UTC" firstStartedPulling="2026-05-26 02:37:49.526166399 +0000 UTC m=+107.082955046" lastFinishedPulling="2026-05-26 02:37:52.591491432 +0000 UTC m=+110.148280082" observedRunningTime="2026-05-26 02:37:53.515649813 +0000 UTC m=+111.072438483" watchObservedRunningTime="2026-05-26 02:37:53.533731168 +0000 UTC m=+111.090519834"
May 26 02:37:54.480451 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:54.480425    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 02:37:54.480939 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:54.480781    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/0.log"
May 26 02:37:54.480939 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:54.480810    2551 generic.go:358] "Generic (PLEG): container finished" podID="fb87f3ac-7bba-4002-8d34-fadbd525b32a" containerID="248bdcfc28b8a3c3419fbeaa2f4f004c8ab554e6e324b69dcd663cda48d39790" exitCode=255
May 26 02:37:54.481059 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:54.480937    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl" event={"ID":"fb87f3ac-7bba-4002-8d34-fadbd525b32a","Type":"ContainerDied","Data":"248bdcfc28b8a3c3419fbeaa2f4f004c8ab554e6e324b69dcd663cda48d39790"}
May 26 02:37:54.481059 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:54.480990    2551 scope.go:117] "RemoveContainer" containerID="db9ed92574de126ff06d8cecdbb8df045123b5dca032c38df3a49bc4c06d5be2"
May 26 02:37:54.481434 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:54.481220    2551 scope.go:117] "RemoveContainer" containerID="248bdcfc28b8a3c3419fbeaa2f4f004c8ab554e6e324b69dcd663cda48d39790"
May 26 02:37:54.481535 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:54.481454    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-5pgsl_openshift-console-operator(fb87f3ac-7bba-4002-8d34-fadbd525b32a)\"" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl" podUID="fb87f3ac-7bba-4002-8d34-fadbd525b32a"
May 26 02:37:55.484353 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:55.484311    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx" event={"ID":"49675dc6-0f22-46e0-a1cd-5408f5598780","Type":"ContainerStarted","Data":"eecdd2b9dcb19f8b7a96ec3849bb6197200803546f96f11ddd2cc304c3d2563f"}
May 26 02:37:55.485752 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:55.485732    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 02:37:55.486062 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:55.486047    2551 scope.go:117] "RemoveContainer" containerID="248bdcfc28b8a3c3419fbeaa2f4f004c8ab554e6e324b69dcd663cda48d39790"
May 26 02:37:55.486209 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:55.486193    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-5pgsl_openshift-console-operator(fb87f3ac-7bba-4002-8d34-fadbd525b32a)\"" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl" podUID="fb87f3ac-7bba-4002-8d34-fadbd525b32a"
May 26 02:37:55.498100 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:55.498060    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx" podStartSLOduration=2.758510262 podStartE2EDuration="4.498047166s" podCreationTimestamp="2026-05-26 02:37:51 +0000 UTC" firstStartedPulling="2026-05-26 02:37:52.790688357 +0000 UTC m=+110.347477005" lastFinishedPulling="2026-05-26 02:37:54.53022526 +0000 UTC m=+112.087013909" observedRunningTime="2026-05-26 02:37:55.497609285 +0000 UTC m=+113.054397976" watchObservedRunningTime="2026-05-26 02:37:55.498047166 +0000 UTC m=+113.054835868"
May 26 02:37:56.102454 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:56.102425    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j55gs_7cef8ed3-f979-4a27-983b-8f89cfa6b7ae/dns-node-resolver/0.log"
May 26 02:37:56.938880 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:56.938841    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:37:56.939363 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:56.939039    2551 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 26 02:37:56.939363 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:56.939066    2551 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-56c484c8cc-pr8sr: secret "image-registry-tls" not found
May 26 02:37:56.939363 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:56.939155    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls podName:a121687e-575e-450a-b679-7ff6c044fc48 nodeName:}" failed. No retries permitted until 2026-05-26 02:38:04.939130792 +0000 UTC m=+122.495919453 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls") pod "image-registry-56c484c8cc-pr8sr" (UID: "a121687e-575e-450a-b679-7ff6c044fc48") : secret "image-registry-tls" not found
May 26 02:37:57.302885 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:57.302861    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q8dr4_7dd95e00-cc75-4517-a4cb-2484530f0fbd/node-ca/0.log"
May 26 02:37:57.853307 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:57.853276    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-67ntt"]
May 26 02:37:57.856245 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:57.856227    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-67ntt"
May 26 02:37:57.858620 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:57.858597    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
May 26 02:37:57.859503 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:57.859481    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-r5bs7\""
May 26 02:37:57.859586 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:57.859530    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
May 26 02:37:57.859586 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:57.859563    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
May 26 02:37:57.859586 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:57.859535    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
May 26 02:37:57.862832 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:57.862537    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-67ntt"]
May 26 02:37:57.945538 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:57.945479    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/30e7f33a-343c-4873-910d-4596b35b3484-signing-cabundle\") pod \"service-ca-67d667c7d-67ntt\" (UID: \"30e7f33a-343c-4873-910d-4596b35b3484\") " pod="openshift-service-ca/service-ca-67d667c7d-67ntt"
May 26 02:37:57.945538 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:57.945539    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/30e7f33a-343c-4873-910d-4596b35b3484-signing-key\") pod \"service-ca-67d667c7d-67ntt\" (UID: \"30e7f33a-343c-4873-910d-4596b35b3484\") " pod="openshift-service-ca/service-ca-67d667c7d-67ntt"
May 26 02:37:57.945989 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:57.945589    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wbd8k\" (UniqueName: \"kubernetes.io/projected/30e7f33a-343c-4873-910d-4596b35b3484-kube-api-access-wbd8k\") pod \"service-ca-67d667c7d-67ntt\" (UID: \"30e7f33a-343c-4873-910d-4596b35b3484\") " pod="openshift-service-ca/service-ca-67d667c7d-67ntt"
May 26 02:37:58.046396 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:58.046340    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/30e7f33a-343c-4873-910d-4596b35b3484-signing-cabundle\") pod \"service-ca-67d667c7d-67ntt\" (UID: \"30e7f33a-343c-4873-910d-4596b35b3484\") " pod="openshift-service-ca/service-ca-67d667c7d-67ntt"
May 26 02:37:58.046396 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:58.046397    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/30e7f33a-343c-4873-910d-4596b35b3484-signing-key\") pod \"service-ca-67d667c7d-67ntt\" (UID: \"30e7f33a-343c-4873-910d-4596b35b3484\") " pod="openshift-service-ca/service-ca-67d667c7d-67ntt"
May 26 02:37:58.046612 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:58.046441    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wbd8k\" (UniqueName: \"kubernetes.io/projected/30e7f33a-343c-4873-910d-4596b35b3484-kube-api-access-wbd8k\") pod \"service-ca-67d667c7d-67ntt\" (UID: \"30e7f33a-343c-4873-910d-4596b35b3484\") " pod="openshift-service-ca/service-ca-67d667c7d-67ntt"
May 26 02:37:58.047009 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:58.046992    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/30e7f33a-343c-4873-910d-4596b35b3484-signing-cabundle\") pod \"service-ca-67d667c7d-67ntt\" (UID: \"30e7f33a-343c-4873-910d-4596b35b3484\") " pod="openshift-service-ca/service-ca-67d667c7d-67ntt"
May 26 02:37:58.048823 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:58.048789    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/30e7f33a-343c-4873-910d-4596b35b3484-signing-key\") pod \"service-ca-67d667c7d-67ntt\" (UID: \"30e7f33a-343c-4873-910d-4596b35b3484\") " pod="openshift-service-ca/service-ca-67d667c7d-67ntt"
May 26 02:37:58.055982 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:58.055953    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wbd8k\" (UniqueName: \"kubernetes.io/projected/30e7f33a-343c-4873-910d-4596b35b3484-kube-api-access-wbd8k\") pod \"service-ca-67d667c7d-67ntt\" (UID: \"30e7f33a-343c-4873-910d-4596b35b3484\") " pod="openshift-service-ca/service-ca-67d667c7d-67ntt"
May 26 02:37:58.165868 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:58.165765    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-67ntt"
May 26 02:37:58.288221 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:58.288162    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-67ntt"]
May 26 02:37:58.292083 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:37:58.292052    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod30e7f33a_343c_4873_910d_4596b35b3484.slice/crio-67a00076367a1e16b18a25524bde868618576e4e54e2eab73471ce149645f65d WatchSource:0}: Error finding container 67a00076367a1e16b18a25524bde868618576e4e54e2eab73471ce149645f65d: Status 404 returned error can't find the container with id 67a00076367a1e16b18a25524bde868618576e4e54e2eab73471ce149645f65d
May 26 02:37:58.496463 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:58.496372    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-67ntt" event={"ID":"30e7f33a-343c-4873-910d-4596b35b3484","Type":"ContainerStarted","Data":"f6ca57dba1c06ebc1757afc6574c01eef2d4f658f8199beb66a4c053372dd1fa"}
May 26 02:37:58.496463 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:58.496416    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-67ntt" event={"ID":"30e7f33a-343c-4873-910d-4596b35b3484","Type":"ContainerStarted","Data":"67a00076367a1e16b18a25524bde868618576e4e54e2eab73471ce149645f65d"}
May 26 02:37:58.511413 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:58.511371    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-67d667c7d-67ntt" podStartSLOduration=1.511357681 podStartE2EDuration="1.511357681s" podCreationTimestamp="2026-05-26 02:37:57 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:37:58.510591373 +0000 UTC m=+116.067380042" watchObservedRunningTime="2026-05-26 02:37:58.511357681 +0000 UTC m=+116.068146401"
May 26 02:37:59.398234 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:59.398191    2551 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:59.398815 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:59.398798    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:37:59.399408 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:37:59.399380    2551 scope.go:117] "RemoveContainer" containerID="248bdcfc28b8a3c3419fbeaa2f4f004c8ab554e6e324b69dcd663cda48d39790"
May 26 02:37:59.399649 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:37:59.399620    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-5pgsl_openshift-console-operator(fb87f3ac-7bba-4002-8d34-fadbd525b32a)\"" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl" podUID="fb87f3ac-7bba-4002-8d34-fadbd525b32a"
May 26 02:38:04.994483 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:04.994443    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:38:04.996782 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:04.996762    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls\") pod \"image-registry-56c484c8cc-pr8sr\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") " pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:38:05.090880 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:05.090852    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-bpf8g\""
May 26 02:38:05.099737 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:05.099715    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:38:05.219744 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:05.219712    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-56c484c8cc-pr8sr"]
May 26 02:38:05.222751 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:38:05.222716    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda121687e_575e_450a_b679_7ff6c044fc48.slice/crio-9719e7b6413d58f74b182e121b1afc1365ee363669586ee699423040bc4cd04f WatchSource:0}: Error finding container 9719e7b6413d58f74b182e121b1afc1365ee363669586ee699423040bc4cd04f: Status 404 returned error can't find the container with id 9719e7b6413d58f74b182e121b1afc1365ee363669586ee699423040bc4cd04f
May 26 02:38:05.516414 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:05.516324    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr" event={"ID":"a121687e-575e-450a-b679-7ff6c044fc48","Type":"ContainerStarted","Data":"4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57"}
May 26 02:38:05.516414 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:05.516367    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr" event={"ID":"a121687e-575e-450a-b679-7ff6c044fc48","Type":"ContainerStarted","Data":"9719e7b6413d58f74b182e121b1afc1365ee363669586ee699423040bc4cd04f"}
May 26 02:38:05.516641 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:05.516526    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:38:05.534699 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:05.534655    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr" podStartSLOduration=16.534641433 podStartE2EDuration="16.534641433s" podCreationTimestamp="2026-05-26 02:37:49 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:38:05.534318063 +0000 UTC m=+123.091106733" watchObservedRunningTime="2026-05-26 02:38:05.534641433 +0000 UTC m=+123.091430104"
May 26 02:38:11.040806 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:11.040770    2551 scope.go:117] "RemoveContainer" containerID="248bdcfc28b8a3c3419fbeaa2f4f004c8ab554e6e324b69dcd663cda48d39790"
May 26 02:38:11.532840 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:11.532814    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 02:38:11.533037 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:11.532871    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl" event={"ID":"fb87f3ac-7bba-4002-8d34-fadbd525b32a","Type":"ContainerStarted","Data":"9276d37f414f709d5589ec198bca5c736987c8bfe676f69e82c81592a626243b"}
May 26 02:38:11.533175 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:11.533152    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:38:11.549625 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:11.549579    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl" podStartSLOduration=19.509720812 podStartE2EDuration="22.549567026s" podCreationTimestamp="2026-05-26 02:37:49 +0000 UTC" firstStartedPulling="2026-05-26 02:37:49.551655167 +0000 UTC m=+107.108443818" lastFinishedPulling="2026-05-26 02:37:52.591501385 +0000 UTC m=+110.148290032" observedRunningTime="2026-05-26 02:38:11.548210274 +0000 UTC m=+129.104998943" watchObservedRunningTime="2026-05-26 02:38:11.549567026 +0000 UTC m=+129.106355695"
May 26 02:38:11.575389 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:11.575358    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-5pgsl"
May 26 02:38:12.757294 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:12.757260    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:38:12.759589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:12.759566    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8-metrics-certs\") pod \"network-metrics-daemon-29jmf\" (UID: \"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8\") " pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:38:12.958283 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:12.958246    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-kjrvv\""
May 26 02:38:12.966277 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:12.966255    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-29jmf"
May 26 02:38:13.108478 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:13.108444    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-29jmf"]
May 26 02:38:13.111945 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:38:13.111914    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9da4a71a_9ddf_4352_9fc7_6cd9f46f35b8.slice/crio-4e6595951775a214c7aebab6875217ffb720ab227183ba1fe440feeddb521a15 WatchSource:0}: Error finding container 4e6595951775a214c7aebab6875217ffb720ab227183ba1fe440feeddb521a15: Status 404 returned error can't find the container with id 4e6595951775a214c7aebab6875217ffb720ab227183ba1fe440feeddb521a15
May 26 02:38:13.539473 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:13.539424    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29jmf" event={"ID":"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8","Type":"ContainerStarted","Data":"4e6595951775a214c7aebab6875217ffb720ab227183ba1fe440feeddb521a15"}
May 26 02:38:14.543124 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:14.543081    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29jmf" event={"ID":"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8","Type":"ContainerStarted","Data":"85c18f6206b70f739c0f52756708758dbc2fd0a73ccab30d46c2065d1c347d7d"}
May 26 02:38:14.543124 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:14.543125    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-29jmf" event={"ID":"9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8","Type":"ContainerStarted","Data":"c1a88bf083f8078c14570e0b9191f108be234963b97572d48c6b1fcc0ed9984f"}
May 26 02:38:14.559409 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:14.559365    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-29jmf" podStartSLOduration=130.663378698 podStartE2EDuration="2m11.55935153s" podCreationTimestamp="2026-05-26 02:36:03 +0000 UTC" firstStartedPulling="2026-05-26 02:38:13.11383944 +0000 UTC m=+130.670628092" lastFinishedPulling="2026-05-26 02:38:14.009812268 +0000 UTC m=+131.566600924" observedRunningTime="2026-05-26 02:38:14.557565779 +0000 UTC m=+132.114354440" watchObservedRunningTime="2026-05-26 02:38:14.55935153 +0000 UTC m=+132.116140237"
May 26 02:38:21.289272 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.289236    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56c484c8cc-pr8sr"]
May 26 02:38:21.293378 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.293338    2551 patch_prober.go:28] interesting pod/image-registry-56c484c8cc-pr8sr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 26 02:38:21.293519 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.293428    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr" podUID="a121687e-575e-450a-b679-7ff6c044fc48" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 26 02:38:21.359648 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.359617    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-jst4x"]
May 26 02:38:21.362462 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.362442    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-jst4x"
May 26 02:38:21.364718 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.364681    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 26 02:38:21.364983 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.364963    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-fg444\""
May 26 02:38:21.365217 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.365196    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 26 02:38:21.375711 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.375688    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-5vxw6"]
May 26 02:38:21.377612 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.377596    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-jst4x"]
May 26 02:38:21.377703 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.377693    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.380544 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.380523    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 26 02:38:21.380651 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.380536    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 26 02:38:21.380651 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.380612    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-8g4gz\""
May 26 02:38:21.396097 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.396035    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5vxw6"]
May 26 02:38:21.415016 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.414988    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/306321fb-101c-4452-a4db-fc8e753b5d53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.415128 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.415021    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-54b64\" (UniqueName: \"kubernetes.io/projected/306321fb-101c-4452-a4db-fc8e753b5d53-kube-api-access-54b64\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.415128 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.415055    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/306321fb-101c-4452-a4db-fc8e753b5d53-data-volume\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.415233 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.415157    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-9nj6w\" (UniqueName: \"kubernetes.io/projected/5f7b0578-f539-440b-9f80-b74ac83d5e09-kube-api-access-9nj6w\") pod \"downloads-57bd96b888-jst4x\" (UID: \"5f7b0578-f539-440b-9f80-b74ac83d5e09\") " pod="openshift-console/downloads-57bd96b888-jst4x"
May 26 02:38:21.415233 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.415191    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/306321fb-101c-4452-a4db-fc8e753b5d53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.415331 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.415245    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/306321fb-101c-4452-a4db-fc8e753b5d53-crio-socket\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.448479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.448450    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-c6ccbcd49-fwcnf"]
May 26 02:38:21.450229 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.450213    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.462983 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.462964    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c6ccbcd49-fwcnf"]
May 26 02:38:21.516605 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.516577    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3895dbfa-efcf-4d00-a0ee-fec2b10af485-registry-tls\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.516605 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.516608    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3895dbfa-efcf-4d00-a0ee-fec2b10af485-registry-certificates\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.516779 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.516626    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3895dbfa-efcf-4d00-a0ee-fec2b10af485-installation-pull-secrets\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.516779 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.516681    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-54b64\" (UniqueName: \"kubernetes.io/projected/306321fb-101c-4452-a4db-fc8e753b5d53-kube-api-access-54b64\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.516779 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.516732    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3895dbfa-efcf-4d00-a0ee-fec2b10af485-image-registry-private-configuration\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.516779 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.516755    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/306321fb-101c-4452-a4db-fc8e753b5d53-data-volume\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.516967 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.516800    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-9nj6w\" (UniqueName: \"kubernetes.io/projected/5f7b0578-f539-440b-9f80-b74ac83d5e09-kube-api-access-9nj6w\") pod \"downloads-57bd96b888-jst4x\" (UID: \"5f7b0578-f539-440b-9f80-b74ac83d5e09\") " pod="openshift-console/downloads-57bd96b888-jst4x"
May 26 02:38:21.516967 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.516818    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/306321fb-101c-4452-a4db-fc8e753b5d53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.516967 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.516835    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/306321fb-101c-4452-a4db-fc8e753b5d53-crio-socket\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.516967 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.516894    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/306321fb-101c-4452-a4db-fc8e753b5d53-crio-socket\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.516967 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.516925    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f6tx5\" (UniqueName: \"kubernetes.io/projected/3895dbfa-efcf-4d00-a0ee-fec2b10af485-kube-api-access-f6tx5\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.517136 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.516983    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3895dbfa-efcf-4d00-a0ee-fec2b10af485-bound-sa-token\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.517136 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.517008    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3895dbfa-efcf-4d00-a0ee-fec2b10af485-trusted-ca\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.517136 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.517049    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3895dbfa-efcf-4d00-a0ee-fec2b10af485-ca-trust-extracted\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.517136 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.517082    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/306321fb-101c-4452-a4db-fc8e753b5d53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.517253 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.517147    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/306321fb-101c-4452-a4db-fc8e753b5d53-data-volume\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.517448 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.517430    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/306321fb-101c-4452-a4db-fc8e753b5d53-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.519304 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.519288    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/306321fb-101c-4452-a4db-fc8e753b5d53-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.525799 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.525778    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-54b64\" (UniqueName: \"kubernetes.io/projected/306321fb-101c-4452-a4db-fc8e753b5d53-kube-api-access-54b64\") pod \"insights-runtime-extractor-5vxw6\" (UID: \"306321fb-101c-4452-a4db-fc8e753b5d53\") " pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.525989 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.525973    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-9nj6w\" (UniqueName: \"kubernetes.io/projected/5f7b0578-f539-440b-9f80-b74ac83d5e09-kube-api-access-9nj6w\") pod \"downloads-57bd96b888-jst4x\" (UID: \"5f7b0578-f539-440b-9f80-b74ac83d5e09\") " pod="openshift-console/downloads-57bd96b888-jst4x"
May 26 02:38:21.617771 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.617736    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f6tx5\" (UniqueName: \"kubernetes.io/projected/3895dbfa-efcf-4d00-a0ee-fec2b10af485-kube-api-access-f6tx5\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.617989 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.617787    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3895dbfa-efcf-4d00-a0ee-fec2b10af485-bound-sa-token\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.617989 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.617844    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3895dbfa-efcf-4d00-a0ee-fec2b10af485-trusted-ca\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.617989 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.617932    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3895dbfa-efcf-4d00-a0ee-fec2b10af485-ca-trust-extracted\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.617989 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.617977    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3895dbfa-efcf-4d00-a0ee-fec2b10af485-registry-tls\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.618209 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.618080    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3895dbfa-efcf-4d00-a0ee-fec2b10af485-registry-certificates\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.618209 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.618111    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3895dbfa-efcf-4d00-a0ee-fec2b10af485-installation-pull-secrets\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.618209 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.618157    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3895dbfa-efcf-4d00-a0ee-fec2b10af485-image-registry-private-configuration\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.618350 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.618314    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/3895dbfa-efcf-4d00-a0ee-fec2b10af485-ca-trust-extracted\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.618880 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.618851    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/3895dbfa-efcf-4d00-a0ee-fec2b10af485-registry-certificates\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.618880 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.618871    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/3895dbfa-efcf-4d00-a0ee-fec2b10af485-trusted-ca\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.620544 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.620523    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/3895dbfa-efcf-4d00-a0ee-fec2b10af485-image-registry-private-configuration\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.620787 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.620769    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/3895dbfa-efcf-4d00-a0ee-fec2b10af485-installation-pull-secrets\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.621036 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.621015    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/3895dbfa-efcf-4d00-a0ee-fec2b10af485-registry-tls\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.626943 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.626921    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f6tx5\" (UniqueName: \"kubernetes.io/projected/3895dbfa-efcf-4d00-a0ee-fec2b10af485-kube-api-access-f6tx5\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.626943 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.626935    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/3895dbfa-efcf-4d00-a0ee-fec2b10af485-bound-sa-token\") pod \"image-registry-c6ccbcd49-fwcnf\" (UID: \"3895dbfa-efcf-4d00-a0ee-fec2b10af485\") " pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.671140 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.671119    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-jst4x"
May 26 02:38:21.686792 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.686773    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-5vxw6"
May 26 02:38:21.758943 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.758885    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:21.800188 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.800155    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-jst4x"]
May 26 02:38:21.803630 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:38:21.803605    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f7b0578_f539_440b_9f80_b74ac83d5e09.slice/crio-10267a56c11b2886bfe019f3991b62f5fbb979542257e9cf2c3cece6f3830c98 WatchSource:0}: Error finding container 10267a56c11b2886bfe019f3991b62f5fbb979542257e9cf2c3cece6f3830c98: Status 404 returned error can't find the container with id 10267a56c11b2886bfe019f3991b62f5fbb979542257e9cf2c3cece6f3830c98
May 26 02:38:21.826302 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.826275    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-5vxw6"]
May 26 02:38:21.829230 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:38:21.829201    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod306321fb_101c_4452_a4db_fc8e753b5d53.slice/crio-0361c581f4597c7f73109e67aecdfd87a547ce6fe38cc52fbd4ba177962ab8ea WatchSource:0}: Error finding container 0361c581f4597c7f73109e67aecdfd87a547ce6fe38cc52fbd4ba177962ab8ea: Status 404 returned error can't find the container with id 0361c581f4597c7f73109e67aecdfd87a547ce6fe38cc52fbd4ba177962ab8ea
May 26 02:38:21.890797 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:21.890769    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-c6ccbcd49-fwcnf"]
May 26 02:38:21.893632 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:38:21.893599    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3895dbfa_efcf_4d00_a0ee_fec2b10af485.slice/crio-b9504d769fbedfd7be4cb1370534c76cf8559bdd21b5a9117c56857adbcb80bd WatchSource:0}: Error finding container b9504d769fbedfd7be4cb1370534c76cf8559bdd21b5a9117c56857adbcb80bd: Status 404 returned error can't find the container with id b9504d769fbedfd7be4cb1370534c76cf8559bdd21b5a9117c56857adbcb80bd
May 26 02:38:22.563887 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:22.563853    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf" event={"ID":"3895dbfa-efcf-4d00-a0ee-fec2b10af485","Type":"ContainerStarted","Data":"b6fde8e23e818ab4e252a160aca02ecbd3751540a37c7b341ca7467fc2e4b674"}
May 26 02:38:22.564270 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:22.563915    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf" event={"ID":"3895dbfa-efcf-4d00-a0ee-fec2b10af485","Type":"ContainerStarted","Data":"b9504d769fbedfd7be4cb1370534c76cf8559bdd21b5a9117c56857adbcb80bd"}
May 26 02:38:22.564270 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:22.564034    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:22.566053 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:22.566027    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5vxw6" event={"ID":"306321fb-101c-4452-a4db-fc8e753b5d53","Type":"ContainerStarted","Data":"c68e6c96a3ed8995cdc91101dc62017059d2c67f447c6125b0c66f69aadeeee0"}
May 26 02:38:22.566168 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:22.566060    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5vxw6" event={"ID":"306321fb-101c-4452-a4db-fc8e753b5d53","Type":"ContainerStarted","Data":"bf372adb6aea345f632af1887d620f527cdbd28e95b555e599cf1ec875746965"}
May 26 02:38:22.566168 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:22.566076    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5vxw6" event={"ID":"306321fb-101c-4452-a4db-fc8e753b5d53","Type":"ContainerStarted","Data":"0361c581f4597c7f73109e67aecdfd87a547ce6fe38cc52fbd4ba177962ab8ea"}
May 26 02:38:22.567111 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:22.567086    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-jst4x" event={"ID":"5f7b0578-f539-440b-9f80-b74ac83d5e09","Type":"ContainerStarted","Data":"10267a56c11b2886bfe019f3991b62f5fbb979542257e9cf2c3cece6f3830c98"}
May 26 02:38:22.583970 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:22.583933    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf" podStartSLOduration=1.583920162 podStartE2EDuration="1.583920162s" podCreationTimestamp="2026-05-26 02:38:21 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:38:22.582329764 +0000 UTC m=+140.139118434" watchObservedRunningTime="2026-05-26 02:38:22.583920162 +0000 UTC m=+140.140708832"
May 26 02:38:24.574824 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:24.574786    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-5vxw6" event={"ID":"306321fb-101c-4452-a4db-fc8e753b5d53","Type":"ContainerStarted","Data":"daa8f039d406ea75184167129f69030f1e1eb9981cf96499c8f316149fc0fb36"}
May 26 02:38:24.595269 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:24.595075    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-5vxw6" podStartSLOduration=1.357510863 podStartE2EDuration="3.595030088s" podCreationTimestamp="2026-05-26 02:38:21 +0000 UTC" firstStartedPulling="2026-05-26 02:38:21.903473565 +0000 UTC m=+139.460262226" lastFinishedPulling="2026-05-26 02:38:24.140992803 +0000 UTC m=+141.697781451" observedRunningTime="2026-05-26 02:38:24.59327951 +0000 UTC m=+142.150068176" watchObservedRunningTime="2026-05-26 02:38:24.595030088 +0000 UTC m=+142.151818760"
May 26 02:38:31.154860 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.154825    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"]
May 26 02:38:31.160524 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.160488    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.163767 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.163738    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 26 02:38:31.164204 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.164020    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-kube-rbac-proxy-config\""
May 26 02:38:31.166626 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.166603    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-zjmg7"]
May 26 02:38:31.170164 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.168715    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 26 02:38:31.170164 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.169283    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-tls\""
May 26 02:38:31.170164 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.169470    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"openshift-state-metrics-dockercfg-5v9xl\""
May 26 02:38:31.189249 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.189226    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 26 02:38:31.192553 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.192151    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"]
May 26 02:38:31.192553 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.192185    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"]
May 26 02:38:31.193169 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.193150    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.196363 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.196340    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"]
May 26 02:38:31.196481 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.196467    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.198652 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.198386    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tmxtc\" (UniqueName: \"kubernetes.io/projected/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-kube-api-access-tmxtc\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.198652 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.198434    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.198652 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.198463    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.198652 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.198568    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.199332 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.199314    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 26 02:38:31.200241 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.199584    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-t462b\""
May 26 02:38:31.201570 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.199651    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 26 02:38:31.201570 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.199720    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 26 02:38:31.201570 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.199780    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
May 26 02:38:31.201570 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.199930    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-9wbh2\""
May 26 02:38:31.201570 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.199964    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
May 26 02:38:31.201570 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.200021    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
May 26 02:38:31.294797 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.294760    2551 patch_prober.go:28] interesting pod/image-registry-56c484c8cc-pr8sr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 26 02:38:31.294990 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.294854    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr" podUID="a121687e-575e-450a-b679-7ff6c044fc48" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 26 02:38:31.299104 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299072    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bdad698-da5b-4759-be30-9011e4728f65-sys\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.299239 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299136    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.299239 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299185    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.299239 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299218    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-tls\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.299402 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299261    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-wtmp\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.299402 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299350    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.299402 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299382    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3bdad698-da5b-4759-be30-9011e4728f65-root\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.299564 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299421    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.299564 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299452    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.299564 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299476    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m9kwk\" (UniqueName: \"kubernetes.io/projected/3bdad698-da5b-4759-be30-9011e4728f65-kube-api-access-m9kwk\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.299564 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299504    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-textfile\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.299564 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299535    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-accelerators-collector-config\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.299793 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299569    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tmxtc\" (UniqueName: \"kubernetes.io/projected/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-kube-api-access-tmxtc\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.299793 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299603    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.299793 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299635    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.299793 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299664    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bdad698-da5b-4759-be30-9011e4728f65-metrics-client-ca\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.299793 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299696    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.299793 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299755    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.300116 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.299808    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w4rws\" (UniqueName: \"kubernetes.io/projected/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-kube-api-access-w4rws\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.300116 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:38:31.299998    2551 secret.go:189] Couldn't get secret openshift-monitoring/openshift-state-metrics-tls: secret "openshift-state-metrics-tls" not found
May 26 02:38:31.300116 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:38:31.300065    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-openshift-state-metrics-tls podName:73274278-9bc8-4b7f-9e3c-2ae1b9a05e62 nodeName:}" failed. No retries permitted until 2026-05-26 02:38:31.800046168 +0000 UTC m=+149.356834820 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "openshift-state-metrics-tls" (UniqueName: "kubernetes.io/secret/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-openshift-state-metrics-tls") pod "openshift-state-metrics-9c959559f-8vthv" (UID: "73274278-9bc8-4b7f-9e3c-2ae1b9a05e62") : secret "openshift-state-metrics-tls" not found
May 26 02:38:31.300462 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.300438    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-metrics-client-ca\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.302390 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.302368    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-openshift-state-metrics-kube-rbac-proxy-config\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.307219 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.307193    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tmxtc\" (UniqueName: \"kubernetes.io/projected/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-kube-api-access-tmxtc\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400270    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bdad698-da5b-4759-be30-9011e4728f65-sys\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400339    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400388    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400422    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-tls\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400448    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-wtmp\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400483    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3bdad698-da5b-4759-be30-9011e4728f65-root\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400517    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400550    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400576    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m9kwk\" (UniqueName: \"kubernetes.io/projected/3bdad698-da5b-4759-be30-9011e4728f65-kube-api-access-m9kwk\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400605    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-textfile\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400633    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-accelerators-collector-config\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400667    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400707    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bdad698-da5b-4759-be30-9011e4728f65-metrics-client-ca\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400737    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.400928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.400776    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w4rws\" (UniqueName: \"kubernetes.io/projected/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-kube-api-access-w4rws\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.402775 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:38:31.402005    2551 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 26 02:38:31.402775 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:38:31.402085    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-tls podName:3bdad698-da5b-4759-be30-9011e4728f65 nodeName:}" failed. No retries permitted until 2026-05-26 02:38:31.902066277 +0000 UTC m=+149.458854928 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-tls") pod "node-exporter-zjmg7" (UID: "3bdad698-da5b-4759-be30-9011e4728f65") : secret "node-exporter-tls" not found
May 26 02:38:31.402775 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.402154    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-accelerators-collector-config\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.402775 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.402173    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/3bdad698-da5b-4759-be30-9011e4728f65-root\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.402775 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.402309    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-wtmp\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.402775 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.402507    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-textfile\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.402775 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.402645    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.402775 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.402695    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/3bdad698-da5b-4759-be30-9011e4728f65-sys\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.402775 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.402732    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/3bdad698-da5b-4759-be30-9011e4728f65-metrics-client-ca\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.403312 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.403151    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.403459 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.403417    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.410120 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.410007    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w4rws\" (UniqueName: \"kubernetes.io/projected/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-kube-api-access-w4rws\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.412499 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.412462    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.412973 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.412894    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.415101 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.415059    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1fc63ba7-ff6b-4fad-8d89-07c470d123f1-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-z5hcp\" (UID: \"1fc63ba7-ff6b-4fad-8d89-07c470d123f1\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.419363 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.419317    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m9kwk\" (UniqueName: \"kubernetes.io/projected/3bdad698-da5b-4759-be30-9011e4728f65-kube-api-access-m9kwk\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.517374 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.517336    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"
May 26 02:38:31.803587 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.803551    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.806313 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.806291    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"openshift-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/73274278-9bc8-4b7f-9e3c-2ae1b9a05e62-openshift-state-metrics-tls\") pod \"openshift-state-metrics-9c959559f-8vthv\" (UID: \"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62\") " pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:31.904747 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.904704    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-tls\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:31.907107 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:31.907077    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/3bdad698-da5b-4759-be30-9011e4728f65-node-exporter-tls\") pod \"node-exporter-zjmg7\" (UID: \"3bdad698-da5b-4759-be30-9011e4728f65\") " pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:32.096864 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:32.096782    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"
May 26 02:38:32.110739 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:32.110712    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-zjmg7"
May 26 02:38:35.474445 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.474410    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/metrics-server-75bc9899bc-v52lw"]
May 26 02:38:35.479380 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.479354    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.482050 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.482027    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-client-certs\""
May 26 02:38:35.483661 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.483278    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-server-audit-profiles\""
May 26 02:38:35.483661 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.483454    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-4phdnd80q9o9b\""
May 26 02:38:35.483924 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.483715    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kubelet-serving-ca-bundle\""
May 26 02:38:35.484023 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.483926    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-tls\""
May 26 02:38:35.484159 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.484107    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"metrics-server-dockercfg-xg6n8\""
May 26 02:38:35.486319 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.486296    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75bc9899bc-v52lw"]
May 26 02:38:35.536353 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.536320    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4d8x5\" (UniqueName: \"kubernetes.io/projected/9f65a295-2483-43b2-95cb-aa7c79471872-kube-api-access-4d8x5\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.536353 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.536360    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9f65a295-2483-43b2-95cb-aa7c79471872-metrics-server-audit-profiles\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.536583 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.536398    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9f65a295-2483-43b2-95cb-aa7c79471872-secret-metrics-server-tls\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.536583 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.536491    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f65a295-2483-43b2-95cb-aa7c79471872-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.536583 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.536549    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9f65a295-2483-43b2-95cb-aa7c79471872-audit-log\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.536716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.536582    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f65a295-2483-43b2-95cb-aa7c79471872-client-ca-bundle\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.536716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.536613    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9f65a295-2483-43b2-95cb-aa7c79471872-secret-metrics-server-client-certs\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.637932 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.637873    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4d8x5\" (UniqueName: \"kubernetes.io/projected/9f65a295-2483-43b2-95cb-aa7c79471872-kube-api-access-4d8x5\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.638089 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.638044    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9f65a295-2483-43b2-95cb-aa7c79471872-metrics-server-audit-profiles\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.638161 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.638101    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9f65a295-2483-43b2-95cb-aa7c79471872-secret-metrics-server-tls\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.638161 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.638136    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f65a295-2483-43b2-95cb-aa7c79471872-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.638262 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.638177    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9f65a295-2483-43b2-95cb-aa7c79471872-audit-log\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.638355 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.638336    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f65a295-2483-43b2-95cb-aa7c79471872-client-ca-bundle\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.638419 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.638382    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9f65a295-2483-43b2-95cb-aa7c79471872-secret-metrics-server-client-certs\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.638564 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.638544    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"audit-log\" (UniqueName: \"kubernetes.io/empty-dir/9f65a295-2483-43b2-95cb-aa7c79471872-audit-log\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.638978 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.638954    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"configmap-kubelet-serving-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/9f65a295-2483-43b2-95cb-aa7c79471872-configmap-kubelet-serving-ca-bundle\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.639202 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.639177    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-server-audit-profiles\" (UniqueName: \"kubernetes.io/configmap/9f65a295-2483-43b2-95cb-aa7c79471872-metrics-server-audit-profiles\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.641045 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.641019    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-tls\" (UniqueName: \"kubernetes.io/secret/9f65a295-2483-43b2-95cb-aa7c79471872-secret-metrics-server-tls\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.641200 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.641183    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"client-ca-bundle\" (UniqueName: \"kubernetes.io/secret/9f65a295-2483-43b2-95cb-aa7c79471872-client-ca-bundle\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.641374 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.641354    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-metrics-server-client-certs\" (UniqueName: \"kubernetes.io/secret/9f65a295-2483-43b2-95cb-aa7c79471872-secret-metrics-server-client-certs\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.644926 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.644885    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4d8x5\" (UniqueName: \"kubernetes.io/projected/9f65a295-2483-43b2-95cb-aa7c79471872-kube-api-access-4d8x5\") pod \"metrics-server-75bc9899bc-v52lw\" (UID: \"9f65a295-2483-43b2-95cb-aa7c79471872\") " pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.796142 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.796061    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:35.953457 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.953424    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld"]
May 26 02:38:35.958071 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.958050    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld"
May 26 02:38:35.960368 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.960340    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
May 26 02:38:35.960488 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.960387    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-m4sgt\""
May 26 02:38:35.963800 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:35.963778    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld"]
May 26 02:38:36.042947 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.042915    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c105fe7-9de3-4666-9ea6-c6fc03128dbf-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-7b5ld\" (UID: \"7c105fe7-9de3-4666-9ea6-c6fc03128dbf\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld"
May 26 02:38:36.144373 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.144332    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c105fe7-9de3-4666-9ea6-c6fc03128dbf-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-7b5ld\" (UID: \"7c105fe7-9de3-4666-9ea6-c6fc03128dbf\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld"
May 26 02:38:36.144531 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:38:36.144507    2551 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
May 26 02:38:36.144607 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:38:36.144595    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7c105fe7-9de3-4666-9ea6-c6fc03128dbf-monitoring-plugin-cert podName:7c105fe7-9de3-4666-9ea6-c6fc03128dbf nodeName:}" failed. No retries permitted until 2026-05-26 02:38:36.644573877 +0000 UTC m=+154.201362528 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/7c105fe7-9de3-4666-9ea6-c6fc03128dbf-monitoring-plugin-cert") pod "monitoring-plugin-665c9c8584-7b5ld" (UID: "7c105fe7-9de3-4666-9ea6-c6fc03128dbf") : secret "monitoring-plugin-cert" not found
May 26 02:38:36.495760 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.495669    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-665b4ffd55-z5xl8"]
May 26 02:38:36.499534 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.499510    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.502170 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.502010    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 26 02:38:36.502170 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.502110    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 26 02:38:36.502170 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.502122    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-ck8sl\""
May 26 02:38:36.502409 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.502126    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 26 02:38:36.502409 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.502032    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 26 02:38:36.502514 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.502443    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 26 02:38:36.506446 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.506395    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-665b4ffd55-z5xl8"]
May 26 02:38:36.507511 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.507478    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
May 26 02:38:36.548484 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.548445    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-config\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.548484 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.548487    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-service-ca\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.548696 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.548561    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-oauth-serving-cert\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.548696 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.548625    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-oauth-config\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.548696 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.548691    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-msspb\" (UniqueName: \"kubernetes.io/projected/d5cffbd0-be2d-4756-a4b2-c07ce183990d-kube-api-access-msspb\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.548832 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.548791    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-trusted-ca-bundle\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.548885 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.548841    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-serving-cert\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.649764 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.649724    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-oauth-config\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.649973 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.649795    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-msspb\" (UniqueName: \"kubernetes.io/projected/d5cffbd0-be2d-4756-a4b2-c07ce183990d-kube-api-access-msspb\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.649973 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.649845    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c105fe7-9de3-4666-9ea6-c6fc03128dbf-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-7b5ld\" (UID: \"7c105fe7-9de3-4666-9ea6-c6fc03128dbf\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld"
May 26 02:38:36.649973 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.649874    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-trusted-ca-bundle\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.650152 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.650093    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-serving-cert\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.650209 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.650168    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-config\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.650258 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.650206    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-service-ca\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.650258 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.650250    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-oauth-serving-cert\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.650970 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.650861    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-config\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.650970 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.650875    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-trusted-ca-bundle\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.651154 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.651060    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-oauth-serving-cert\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.651154 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.651138    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-service-ca\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.652854 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.652828    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-serving-cert\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.653133 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.653105    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-oauth-config\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.653199 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.653137    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/7c105fe7-9de3-4666-9ea6-c6fc03128dbf-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-7b5ld\" (UID: \"7c105fe7-9de3-4666-9ea6-c6fc03128dbf\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld"
May 26 02:38:36.657322 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.657296    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-msspb\" (UniqueName: \"kubernetes.io/projected/d5cffbd0-be2d-4756-a4b2-c07ce183990d-kube-api-access-msspb\") pod \"console-665b4ffd55-z5xl8\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") " pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.811914 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.811854    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:36.870343 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:36.870307    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld"
May 26 02:38:37.508673 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.508647    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/openshift-state-metrics-9c959559f-8vthv"]
May 26 02:38:37.520525 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:38:37.520480    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod73274278_9bc8_4b7f_9e3c_2ae1b9a05e62.slice/crio-12268cb8d68567c841e3953bf5825a18a6d55321ec17841249693924689c381c WatchSource:0}: Error finding container 12268cb8d68567c841e3953bf5825a18a6d55321ec17841249693924689c381c: Status 404 returned error can't find the container with id 12268cb8d68567c841e3953bf5825a18a6d55321ec17841249693924689c381c
May 26 02:38:37.590068 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.590032    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-665b4ffd55-z5xl8"]
May 26 02:38:37.594397 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:38:37.594363    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd5cffbd0_be2d_4756_a4b2_c07ce183990d.slice/crio-45dece4b3049684d31b9633f3dbd73113b36a4c4ec1e53b7786cd3cb35d9e65b WatchSource:0}: Error finding container 45dece4b3049684d31b9633f3dbd73113b36a4c4ec1e53b7786cd3cb35d9e65b: Status 404 returned error can't find the container with id 45dece4b3049684d31b9633f3dbd73113b36a4c4ec1e53b7786cd3cb35d9e65b
May 26 02:38:37.613545 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.613512    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-665b4ffd55-z5xl8" event={"ID":"d5cffbd0-be2d-4756-a4b2-c07ce183990d","Type":"ContainerStarted","Data":"45dece4b3049684d31b9633f3dbd73113b36a4c4ec1e53b7786cd3cb35d9e65b"}
May 26 02:38:37.615990 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.615062    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv" event={"ID":"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62","Type":"ContainerStarted","Data":"69e1272ac0768a1b36788d7fdb52bd642e7ab4841cc71d027e95fae0a642b61d"}
May 26 02:38:37.615990 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.615108    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv" event={"ID":"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62","Type":"ContainerStarted","Data":"12268cb8d68567c841e3953bf5825a18a6d55321ec17841249693924689c381c"}
May 26 02:38:37.617954 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.616534    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-jst4x" event={"ID":"5f7b0578-f539-440b-9f80-b74ac83d5e09","Type":"ContainerStarted","Data":"fa4e58893aed6dbd911727efa21fe15f9dde1bdd8481df969d2205b4db02f2da"}
May 26 02:38:37.617954 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.617147    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-jst4x"
May 26 02:38:37.617954 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.617621    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjmg7" event={"ID":"3bdad698-da5b-4759-be30-9011e4728f65","Type":"ContainerStarted","Data":"3b7995bcac035e186fe2679f85f3b411c30e1cc43a7026bcc3653c015a6463b0"}
May 26 02:38:37.618458 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.618440    2551 patch_prober.go:28] interesting pod/downloads-57bd96b888-jst4x container/download-server namespace/openshift-console: Readiness probe status=failure output="Get \"http://10.134.0.14:8080/\": dial tcp 10.134.0.14:8080: connect: connection refused" start-of-body=
May 26 02:38:37.618586 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.618545    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/downloads-57bd96b888-jst4x" podUID="5f7b0578-f539-440b-9f80-b74ac83d5e09" containerName="download-server" probeResult="failure" output="Get \"http://10.134.0.14:8080/\": dial tcp 10.134.0.14:8080: connect: connection refused"
May 26 02:38:37.633119 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.633028    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-jst4x" podStartSLOduration=0.97059489 podStartE2EDuration="16.63301697s" podCreationTimestamp="2026-05-26 02:38:21 +0000 UTC" firstStartedPulling="2026-05-26 02:38:21.806134825 +0000 UTC m=+139.362923476" lastFinishedPulling="2026-05-26 02:38:37.468556909 +0000 UTC m=+155.025345556" observedRunningTime="2026-05-26 02:38:37.63171799 +0000 UTC m=+155.188506671" watchObservedRunningTime="2026-05-26 02:38:37.63301697 +0000 UTC m=+155.189805638"
May 26 02:38:37.733869 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.733637    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp"]
May 26 02:38:37.737058 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.737031    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld"]
May 26 02:38:37.737483 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:38:37.737441    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1fc63ba7_ff6b_4fad_8d89_07c470d123f1.slice/crio-2cf213c103bbf2876419eeab246a09dd9c93bb860f7fe8da083cbb5d72eb938c WatchSource:0}: Error finding container 2cf213c103bbf2876419eeab246a09dd9c93bb860f7fe8da083cbb5d72eb938c: Status 404 returned error can't find the container with id 2cf213c103bbf2876419eeab246a09dd9c93bb860f7fe8da083cbb5d72eb938c
May 26 02:38:37.739563 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:37.739372    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/metrics-server-75bc9899bc-v52lw"]
May 26 02:38:37.742559 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:38:37.742507    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7c105fe7_9de3_4666_9ea6_c6fc03128dbf.slice/crio-021744b8297e12871f2480503fbbf8ba5ff05cb9fd002bfca1b4e59c103f3de7 WatchSource:0}: Error finding container 021744b8297e12871f2480503fbbf8ba5ff05cb9fd002bfca1b4e59c103f3de7: Status 404 returned error can't find the container with id 021744b8297e12871f2480503fbbf8ba5ff05cb9fd002bfca1b4e59c103f3de7
May 26 02:38:37.744547 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:38:37.744509    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod9f65a295_2483_43b2_95cb_aa7c79471872.slice/crio-778597b8121435d96e7b8d5710821932954ccc17466e649143632a872eee6aa1 WatchSource:0}: Error finding container 778597b8121435d96e7b8d5710821932954ccc17466e649143632a872eee6aa1: Status 404 returned error can't find the container with id 778597b8121435d96e7b8d5710821932954ccc17466e649143632a872eee6aa1
May 26 02:38:38.633552 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:38.633495    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv" event={"ID":"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62","Type":"ContainerStarted","Data":"c2523838b726b472a8dd59394f2895bae58e366d0e01011bcc72f33ef8ced92f"}
May 26 02:38:38.635603 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:38.635565    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp" event={"ID":"1fc63ba7-ff6b-4fad-8d89-07c470d123f1","Type":"ContainerStarted","Data":"2cf213c103bbf2876419eeab246a09dd9c93bb860f7fe8da083cbb5d72eb938c"}
May 26 02:38:38.638421 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:38.638370    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld" event={"ID":"7c105fe7-9de3-4666-9ea6-c6fc03128dbf","Type":"ContainerStarted","Data":"021744b8297e12871f2480503fbbf8ba5ff05cb9fd002bfca1b4e59c103f3de7"}
May 26 02:38:38.641733 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:38.641576    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw" event={"ID":"9f65a295-2483-43b2-95cb-aa7c79471872","Type":"ContainerStarted","Data":"778597b8121435d96e7b8d5710821932954ccc17466e649143632a872eee6aa1"}
May 26 02:38:38.659337 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:38.659218    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-jst4x"
May 26 02:38:38.793008 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:38:38.792951    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-r4986" podUID="a7e61c19-3818-47b3-82b7-bc5b51bb09de"
May 26 02:38:38.814196 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:38:38.813113    2551 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-tpfbl" podUID="96779027-bcda-438b-924c-3441dd3c4b9d"
May 26 02:38:39.645989 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:39.645591    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r4986"
May 26 02:38:41.294676 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:41.294561    2551 patch_prober.go:28] interesting pod/image-registry-56c484c8cc-pr8sr container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 26 02:38:41.294676 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:41.294620    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr" podUID="a121687e-575e-450a-b679-7ff6c044fc48" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 26 02:38:41.765187 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:41.765151    2551 patch_prober.go:28] interesting pod/image-registry-c6ccbcd49-fwcnf container/registry namespace/openshift-image-registry: Liveness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 26 02:38:41.765381 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:41.765211    2551 prober.go:120] "Probe failed" probeType="Liveness" pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf" podUID="3895dbfa-efcf-4d00-a0ee-fec2b10af485" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 26 02:38:43.576096 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.576067    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-c6ccbcd49-fwcnf"
May 26 02:38:43.628348 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.627608    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:38:43.631079 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.631018    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/a7e61c19-3818-47b3-82b7-bc5b51bb09de-metrics-tls\") pod \"dns-default-r4986\" (UID: \"a7e61c19-3818-47b3-82b7-bc5b51bb09de\") " pod="openshift-dns/dns-default-r4986"
May 26 02:38:43.669957 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.669101    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld" event={"ID":"7c105fe7-9de3-4666-9ea6-c6fc03128dbf","Type":"ContainerStarted","Data":"e0d518b717feae99546f53463ab408e546d34699c49bd0feb3c969fdb215f679"}
May 26 02:38:43.672862 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.670521    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw" event={"ID":"9f65a295-2483-43b2-95cb-aa7c79471872","Type":"ContainerStarted","Data":"97c5c5a302c77bc801e66cde6f74cc1d8aeb1c370c3641f994b61060c3c9644e"}
May 26 02:38:43.672862 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.672029    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-665b4ffd55-z5xl8" event={"ID":"d5cffbd0-be2d-4756-a4b2-c07ce183990d","Type":"ContainerStarted","Data":"a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f"}
May 26 02:38:43.689309 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.679672    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld"
May 26 02:38:43.689309 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.680591    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv" event={"ID":"73274278-9bc8-4b7f-9e3c-2ae1b9a05e62","Type":"ContainerStarted","Data":"08adaa96ca0ea9f715d696716a0134415115b4285254dadf54881a629b021d2b"}
May 26 02:38:43.689309 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.682563    2551 generic.go:358] "Generic (PLEG): container finished" podID="3bdad698-da5b-4759-be30-9011e4728f65" containerID="7351553ffc1a3c0b30b0795ee9e323419fcf04235f87523c3f60bd445b9ebda8" exitCode=0
May 26 02:38:43.689309 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.682625    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjmg7" event={"ID":"3bdad698-da5b-4759-be30-9011e4728f65","Type":"ContainerDied","Data":"7351553ffc1a3c0b30b0795ee9e323419fcf04235f87523c3f60bd445b9ebda8"}
May 26 02:38:43.689309 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.684890    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp" event={"ID":"1fc63ba7-ff6b-4fad-8d89-07c470d123f1","Type":"ContainerStarted","Data":"178fd45bc8eb817c12f7f51e228576a327619219df9bc4247d1a88313e6a3b16"}
May 26 02:38:43.689309 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.684936    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp" event={"ID":"1fc63ba7-ff6b-4fad-8d89-07c470d123f1","Type":"ContainerStarted","Data":"729d44baf466d3ba725efcd63028a0a4d654c4209e002b81bc675e67a7dea6a1"}
May 26 02:38:43.696012 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.695964    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld" podStartSLOduration=3.2738518819999998 podStartE2EDuration="8.695943448s" podCreationTimestamp="2026-05-26 02:38:35 +0000 UTC" firstStartedPulling="2026-05-26 02:38:37.744835236 +0000 UTC m=+155.301623882" lastFinishedPulling="2026-05-26 02:38:43.166926788 +0000 UTC m=+160.723715448" observedRunningTime="2026-05-26 02:38:43.694976134 +0000 UTC m=+161.251764817" watchObservedRunningTime="2026-05-26 02:38:43.695943448 +0000 UTC m=+161.252732117"
May 26 02:38:43.705942 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.705344    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-7b5ld"
May 26 02:38:43.723890 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.723323    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-665b4ffd55-z5xl8" podStartSLOduration=2.145158261 podStartE2EDuration="7.723301553s" podCreationTimestamp="2026-05-26 02:38:36 +0000 UTC" firstStartedPulling="2026-05-26 02:38:37.5963495 +0000 UTC m=+155.153138147" lastFinishedPulling="2026-05-26 02:38:43.174492788 +0000 UTC m=+160.731281439" observedRunningTime="2026-05-26 02:38:43.722654843 +0000 UTC m=+161.279443512" watchObservedRunningTime="2026-05-26 02:38:43.723301553 +0000 UTC m=+161.280090224"
May 26 02:38:43.728676 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.728646    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:38:43.734951 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.734877    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/96779027-bcda-438b-924c-3441dd3c4b9d-cert\") pod \"ingress-canary-tpfbl\" (UID: \"96779027-bcda-438b-924c-3441dd3c4b9d\") " pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:38:43.792349 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.791551    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw" podStartSLOduration=3.371815703 podStartE2EDuration="8.791533216s" podCreationTimestamp="2026-05-26 02:38:35 +0000 UTC" firstStartedPulling="2026-05-26 02:38:37.746699404 +0000 UTC m=+155.303488050" lastFinishedPulling="2026-05-26 02:38:43.166416907 +0000 UTC m=+160.723205563" observedRunningTime="2026-05-26 02:38:43.778710571 +0000 UTC m=+161.335499242" watchObservedRunningTime="2026-05-26 02:38:43.791533216 +0000 UTC m=+161.348321888"
May 26 02:38:43.812402 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.811144    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/openshift-state-metrics-9c959559f-8vthv" podStartSLOduration=7.278234063 podStartE2EDuration="12.811113751s" podCreationTimestamp="2026-05-26 02:38:31 +0000 UTC" firstStartedPulling="2026-05-26 02:38:37.632773938 +0000 UTC m=+155.189562598" lastFinishedPulling="2026-05-26 02:38:43.165653628 +0000 UTC m=+160.722442286" observedRunningTime="2026-05-26 02:38:43.808261708 +0000 UTC m=+161.365050378" watchObservedRunningTime="2026-05-26 02:38:43.811113751 +0000 UTC m=+161.367902467"
May 26 02:38:43.848952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.848736    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-77jkf\""
May 26 02:38:43.856723 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:43.856699    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-r4986"
May 26 02:38:44.029075 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:44.029016    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-r4986"]
May 26 02:38:44.690823 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:44.690741    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjmg7" event={"ID":"3bdad698-da5b-4759-be30-9011e4728f65","Type":"ContainerStarted","Data":"6c1da3e257a00efae6fda7404e55215049f262bfe30cf98c7175d5b350e58163"}
May 26 02:38:44.690823 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:44.690796    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-zjmg7" event={"ID":"3bdad698-da5b-4759-be30-9011e4728f65","Type":"ContainerStarted","Data":"f954d8c269670e9a43bf117792197bfea924f43553be87ed079ca3d8d78f3fa3"}
May 26 02:38:44.692607 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:44.692577    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r4986" event={"ID":"a7e61c19-3818-47b3-82b7-bc5b51bb09de","Type":"ContainerStarted","Data":"bc399199e2378a36d8ebd5c41ac05de71bdc870086ec471e0bad721f047d403e"}
May 26 02:38:44.695398 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:44.695359    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp" event={"ID":"1fc63ba7-ff6b-4fad-8d89-07c470d123f1","Type":"ContainerStarted","Data":"c5a9ef6ff53bd74670bc3ba0fe8f440a0c9a91a87772ff67fa8b240a356c53ba"}
May 26 02:38:44.712476 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:44.712230    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-zjmg7" podStartSLOduration=7.944381666 podStartE2EDuration="13.712213937s" podCreationTimestamp="2026-05-26 02:38:31 +0000 UTC" firstStartedPulling="2026-05-26 02:38:37.368671288 +0000 UTC m=+154.925459939" lastFinishedPulling="2026-05-26 02:38:43.136503563 +0000 UTC m=+160.693292210" observedRunningTime="2026-05-26 02:38:44.711265643 +0000 UTC m=+162.268054315" watchObservedRunningTime="2026-05-26 02:38:44.712213937 +0000 UTC m=+162.269002607"
May 26 02:38:44.729764 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:44.729709    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-z5hcp" podStartSLOduration=8.015737099 podStartE2EDuration="13.729695818s" podCreationTimestamp="2026-05-26 02:38:31 +0000 UTC" firstStartedPulling="2026-05-26 02:38:37.739633949 +0000 UTC m=+155.296422615" lastFinishedPulling="2026-05-26 02:38:43.453592669 +0000 UTC m=+161.010381334" observedRunningTime="2026-05-26 02:38:44.727282083 +0000 UTC m=+162.284070753" watchObservedRunningTime="2026-05-26 02:38:44.729695818 +0000 UTC m=+162.286484488"
May 26 02:38:46.307505 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.307456    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr" podUID="a121687e-575e-450a-b679-7ff6c044fc48" containerName="registry" containerID="cri-o://4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57" gracePeriod=30
May 26 02:38:46.562171 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.562149    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:38:46.661006 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.660962    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a121687e-575e-450a-b679-7ff6c044fc48-installation-pull-secrets\") pod \"a121687e-575e-450a-b679-7ff6c044fc48\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") "
May 26 02:38:46.661210 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.661051    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a121687e-575e-450a-b679-7ff6c044fc48-registry-certificates\") pod \"a121687e-575e-450a-b679-7ff6c044fc48\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") "
May 26 02:38:46.661210 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.661089    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls\") pod \"a121687e-575e-450a-b679-7ff6c044fc48\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") "
May 26 02:38:46.661210 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.661123    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-csh5x\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-kube-api-access-csh5x\") pod \"a121687e-575e-450a-b679-7ff6c044fc48\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") "
May 26 02:38:46.661376 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.661347    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a121687e-575e-450a-b679-7ff6c044fc48-trusted-ca\") pod \"a121687e-575e-450a-b679-7ff6c044fc48\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") "
May 26 02:38:46.661488 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.661467    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a121687e-575e-450a-b679-7ff6c044fc48-image-registry-private-configuration\") pod \"a121687e-575e-450a-b679-7ff6c044fc48\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") "
May 26 02:38:46.661563 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.661516    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a121687e-575e-450a-b679-7ff6c044fc48-ca-trust-extracted\") pod \"a121687e-575e-450a-b679-7ff6c044fc48\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") "
May 26 02:38:46.661563 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.661517    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a121687e-575e-450a-b679-7ff6c044fc48-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "a121687e-575e-450a-b679-7ff6c044fc48" (UID: "a121687e-575e-450a-b679-7ff6c044fc48"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:38:46.661563 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.661556    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-bound-sa-token\") pod \"a121687e-575e-450a-b679-7ff6c044fc48\" (UID: \"a121687e-575e-450a-b679-7ff6c044fc48\") "
May 26 02:38:46.661865 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.661834    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a121687e-575e-450a-b679-7ff6c044fc48-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "a121687e-575e-450a-b679-7ff6c044fc48" (UID: "a121687e-575e-450a-b679-7ff6c044fc48"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:38:46.661966 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.661892    2551 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/a121687e-575e-450a-b679-7ff6c044fc48-registry-certificates\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:38:46.664215 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.664185    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a121687e-575e-450a-b679-7ff6c044fc48-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "a121687e-575e-450a-b679-7ff6c044fc48" (UID: "a121687e-575e-450a-b679-7ff6c044fc48"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:38:46.664321 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.664212    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a121687e-575e-450a-b679-7ff6c044fc48-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "a121687e-575e-450a-b679-7ff6c044fc48" (UID: "a121687e-575e-450a-b679-7ff6c044fc48"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:38:46.664395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.664321    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-kube-api-access-csh5x" (OuterVolumeSpecName: "kube-api-access-csh5x") pod "a121687e-575e-450a-b679-7ff6c044fc48" (UID: "a121687e-575e-450a-b679-7ff6c044fc48"). InnerVolumeSpecName "kube-api-access-csh5x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:38:46.664395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.664334    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a121687e-575e-450a-b679-7ff6c044fc48" (UID: "a121687e-575e-450a-b679-7ff6c044fc48"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:38:46.664505 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.664403    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "a121687e-575e-450a-b679-7ff6c044fc48" (UID: "a121687e-575e-450a-b679-7ff6c044fc48"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:38:46.671745 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.671718    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a121687e-575e-450a-b679-7ff6c044fc48-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "a121687e-575e-450a-b679-7ff6c044fc48" (UID: "a121687e-575e-450a-b679-7ff6c044fc48"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 02:38:46.703521 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.703445    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r4986" event={"ID":"a7e61c19-3818-47b3-82b7-bc5b51bb09de","Type":"ContainerStarted","Data":"3f03ed2c6176ec7f8def409068abd6b946b12803e20fe30233cf05eed808fc38"}
May 26 02:38:46.703521 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.703482    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-r4986" event={"ID":"a7e61c19-3818-47b3-82b7-bc5b51bb09de","Type":"ContainerStarted","Data":"8728ba8345428616ecf66adac8005c8f4567842469dedf2a679405c272bddd1e"}
May 26 02:38:46.703675 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.703560    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-r4986"
May 26 02:38:46.704489 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.704462    2551 generic.go:358] "Generic (PLEG): container finished" podID="a121687e-575e-450a-b679-7ff6c044fc48" containerID="4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57" exitCode=0
May 26 02:38:46.704587 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.704509    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr" event={"ID":"a121687e-575e-450a-b679-7ff6c044fc48","Type":"ContainerDied","Data":"4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57"}
May 26 02:38:46.704587 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.704526    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr" event={"ID":"a121687e-575e-450a-b679-7ff6c044fc48","Type":"ContainerDied","Data":"9719e7b6413d58f74b182e121b1afc1365ee363669586ee699423040bc4cd04f"}
May 26 02:38:46.704587 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.704542    2551 scope.go:117] "RemoveContainer" containerID="4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57"
May 26 02:38:46.704587 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.704548    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-56c484c8cc-pr8sr"
May 26 02:38:46.712738 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.712720    2551 scope.go:117] "RemoveContainer" containerID="4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57"
May 26 02:38:46.713006 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:38:46.712987    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57\": container with ID starting with 4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57 not found: ID does not exist" containerID="4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57"
May 26 02:38:46.713075 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.713012    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57"} err="failed to get container status \"4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57\": rpc error: code = NotFound desc = could not find container \"4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57\": container with ID starting with 4d01812d90002204afb036271bc03f2b2a0402cf2282b194d72416cd429eba57 not found: ID does not exist"
May 26 02:38:46.720182 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.720144    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-r4986" podStartSLOduration=129.865973304 podStartE2EDuration="2m11.720133287s" podCreationTimestamp="2026-05-26 02:36:35 +0000 UTC" firstStartedPulling="2026-05-26 02:38:44.035956473 +0000 UTC m=+161.592745128" lastFinishedPulling="2026-05-26 02:38:45.890116445 +0000 UTC m=+163.446905111" observedRunningTime="2026-05-26 02:38:46.718866803 +0000 UTC m=+164.275655473" watchObservedRunningTime="2026-05-26 02:38:46.720133287 +0000 UTC m=+164.276921956"
May 26 02:38:46.731133 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.731044    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-56c484c8cc-pr8sr"]
May 26 02:38:46.732521 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.732498    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-56c484c8cc-pr8sr"]
May 26 02:38:46.762641 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.762613    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/a121687e-575e-450a-b679-7ff6c044fc48-trusted-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:38:46.762641 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.762636    2551 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/a121687e-575e-450a-b679-7ff6c044fc48-image-registry-private-configuration\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:38:46.762790 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.762648    2551 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/a121687e-575e-450a-b679-7ff6c044fc48-ca-trust-extracted\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:38:46.762790 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.762660    2551 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-bound-sa-token\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:38:46.762790 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.762674    2551 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/a121687e-575e-450a-b679-7ff6c044fc48-installation-pull-secrets\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:38:46.762790 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.762688    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:38:46.762790 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.762702    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-csh5x\" (UniqueName: \"kubernetes.io/projected/a121687e-575e-450a-b679-7ff6c044fc48-kube-api-access-csh5x\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:38:46.812478 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.812449    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:46.812478 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.812480    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:46.817395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:46.817367    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:47.044812 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:47.044718    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a121687e-575e-450a-b679-7ff6c044fc48" path="/var/lib/kubelet/pods/a121687e-575e-450a-b679-7ff6c044fc48/volumes"
May 26 02:38:47.712704 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:47.712674    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:38:50.040740 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:50.040702    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:38:50.043272 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:50.043255    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-jxvlq\""
May 26 02:38:50.051929 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:50.051478    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-tpfbl"
May 26 02:38:50.174497 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:50.174466    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-tpfbl"]
May 26 02:38:50.177961 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:38:50.177931    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod96779027_bcda_438b_924c_3441dd3c4b9d.slice/crio-a50716551ca5d72b2000669d18a4f161911ddd7e473dcdbfbd2da58c75742e3d WatchSource:0}: Error finding container a50716551ca5d72b2000669d18a4f161911ddd7e473dcdbfbd2da58c75742e3d: Status 404 returned error can't find the container with id a50716551ca5d72b2000669d18a4f161911ddd7e473dcdbfbd2da58c75742e3d
May 26 02:38:50.724086 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:50.724048    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tpfbl" event={"ID":"96779027-bcda-438b-924c-3441dd3c4b9d","Type":"ContainerStarted","Data":"a50716551ca5d72b2000669d18a4f161911ddd7e473dcdbfbd2da58c75742e3d"}
May 26 02:38:52.731141 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:52.731064    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-tpfbl" event={"ID":"96779027-bcda-438b-924c-3441dd3c4b9d","Type":"ContainerStarted","Data":"c9c2282403acbad948c7333a5f94ea07be0f8dfc6ca6f28dc550beeacd5ee118"}
May 26 02:38:52.750666 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:52.750617    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-tpfbl" podStartSLOduration=135.550180122 podStartE2EDuration="2m17.750602678s" podCreationTimestamp="2026-05-26 02:36:35 +0000 UTC" firstStartedPulling="2026-05-26 02:38:50.179738762 +0000 UTC m=+167.736527410" lastFinishedPulling="2026-05-26 02:38:52.380161304 +0000 UTC m=+169.936949966" observedRunningTime="2026-05-26 02:38:52.750078888 +0000 UTC m=+170.306867558" watchObservedRunningTime="2026-05-26 02:38:52.750602678 +0000 UTC m=+170.307391346"
May 26 02:38:55.796756 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:55.796726    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:55.796756 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:55.796763    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:38:56.711101 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:56.711068    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-r4986"
May 26 02:38:58.749149 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:58.749113    2551 generic.go:358] "Generic (PLEG): container finished" podID="355b4827-cbde-4c11-813c-08ed3b73ad7d" containerID="c88d6c54a5fc2e00b036d9e430c0149f598dc68e6aa3af21385834f21fea03fa" exitCode=0
May 26 02:38:58.749522 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:58.749188    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj" event={"ID":"355b4827-cbde-4c11-813c-08ed3b73ad7d","Type":"ContainerDied","Data":"c88d6c54a5fc2e00b036d9e430c0149f598dc68e6aa3af21385834f21fea03fa"}
May 26 02:38:58.749570 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:58.749520    2551 scope.go:117] "RemoveContainer" containerID="c88d6c54a5fc2e00b036d9e430c0149f598dc68e6aa3af21385834f21fea03fa"
May 26 02:38:59.753801 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:38:59.753763    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-xs5pj" event={"ID":"355b4827-cbde-4c11-813c-08ed3b73ad7d","Type":"ContainerStarted","Data":"9431b942bcb318ccbf6a38c2a84f925c9fda48117876cd1cb2e2ebd7b138d4fa"}
May 26 02:39:08.781006 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:08.780920    2551 generic.go:358] "Generic (PLEG): container finished" podID="1511080f-2218-4a2b-8395-1ae813c119b2" containerID="e201c7740d7c4d8750f1005ba291f474b695b8cd89fea502494aa6a109793afa" exitCode=0
May 26 02:39:08.781006 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:08.780987    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-p9x4q" event={"ID":"1511080f-2218-4a2b-8395-1ae813c119b2","Type":"ContainerDied","Data":"e201c7740d7c4d8750f1005ba291f474b695b8cd89fea502494aa6a109793afa"}
May 26 02:39:08.781395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:08.781329    2551 scope.go:117] "RemoveContainer" containerID="e201c7740d7c4d8750f1005ba291f474b695b8cd89fea502494aa6a109793afa"
May 26 02:39:09.785642 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:09.785607    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-operator-78bdfb888c-p9x4q" event={"ID":"1511080f-2218-4a2b-8395-1ae813c119b2","Type":"ContainerStarted","Data":"6f7e2767ae124a69ebf5c5c0efbf12e37b2e4d0df26995e98d4e359336003382"}
May 26 02:39:14.800450 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:14.800414    2551 generic.go:358] "Generic (PLEG): container finished" podID="49675dc6-0f22-46e0-a1cd-5408f5598780" containerID="eecdd2b9dcb19f8b7a96ec3849bb6197200803546f96f11ddd2cc304c3d2563f" exitCode=0
May 26 02:39:14.800948 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:14.800490    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx" event={"ID":"49675dc6-0f22-46e0-a1cd-5408f5598780","Type":"ContainerDied","Data":"eecdd2b9dcb19f8b7a96ec3849bb6197200803546f96f11ddd2cc304c3d2563f"}
May 26 02:39:14.801028 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:14.800949    2551 scope.go:117] "RemoveContainer" containerID="eecdd2b9dcb19f8b7a96ec3849bb6197200803546f96f11ddd2cc304c3d2563f"
May 26 02:39:15.802795 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:15.802730    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:39:15.805165 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:15.805136    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-wptxx" event={"ID":"49675dc6-0f22-46e0-a1cd-5408f5598780","Type":"ContainerStarted","Data":"9c52929bfbd1ce54ba07f5be9dbcbda995221ec95c39c2adaa70a60831ba5ae2"}
May 26 02:39:15.806986 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:15.806968    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/metrics-server-75bc9899bc-v52lw"
May 26 02:39:55.448760 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.448722    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"]
May 26 02:39:55.449364 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.449152    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a121687e-575e-450a-b679-7ff6c044fc48" containerName="registry"
May 26 02:39:55.449364 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.449168    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a121687e-575e-450a-b679-7ff6c044fc48" containerName="registry"
May 26 02:39:55.449364 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.449304    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="a121687e-575e-450a-b679-7ff6c044fc48" containerName="registry"
May 26 02:39:55.451567 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.451547    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.454011 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.453988    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client\""
May 26 02:39:55.454282 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.454268    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-dockercfg-s47fp\""
May 26 02:39:55.454331 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.454295    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"federate-client-certs\""
May 26 02:39:55.454533 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.454516    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-client-serving-certs-ca-bundle\""
May 26 02:39:55.455027 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.455011    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-tls\""
May 26 02:39:55.455121 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.455011    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"telemeter-client-kube-rbac-proxy-config\""
May 26 02:39:55.458531 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.458511    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"telemeter-trusted-ca-bundle-8i12ta5c71j38\""
May 26 02:39:55.462517 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.462489    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"]
May 26 02:39:55.538655 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.538624    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0ef0b8fc-02e3-44f3-9b8d-54740392b572-secret-telemeter-client\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.538807 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.538670    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ef0b8fc-02e3-44f3-9b8d-54740392b572-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.538807 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.538728    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-xt6c4\" (UniqueName: \"kubernetes.io/projected/0ef0b8fc-02e3-44f3-9b8d-54740392b572-kube-api-access-xt6c4\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.538807 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.538781    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0ef0b8fc-02e3-44f3-9b8d-54740392b572-telemeter-client-tls\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.538949 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.538811    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0ef0b8fc-02e3-44f3-9b8d-54740392b572-federate-client-tls\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.538949 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.538834    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ef0b8fc-02e3-44f3-9b8d-54740392b572-metrics-client-ca\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.538949 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.538917    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef0b8fc-02e3-44f3-9b8d-54740392b572-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.539041 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.538998    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef0b8fc-02e3-44f3-9b8d-54740392b572-serving-certs-ca-bundle\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.639373 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.639336    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0ef0b8fc-02e3-44f3-9b8d-54740392b572-telemeter-client-tls\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.639373 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.639372    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0ef0b8fc-02e3-44f3-9b8d-54740392b572-federate-client-tls\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.639569 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.639395    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ef0b8fc-02e3-44f3-9b8d-54740392b572-metrics-client-ca\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.639613 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.639579    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef0b8fc-02e3-44f3-9b8d-54740392b572-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.639660 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.639639    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef0b8fc-02e3-44f3-9b8d-54740392b572-serving-certs-ca-bundle\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.639723 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.639671    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0ef0b8fc-02e3-44f3-9b8d-54740392b572-secret-telemeter-client\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.639781 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.639735    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ef0b8fc-02e3-44f3-9b8d-54740392b572-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.639781 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.639769    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-xt6c4\" (UniqueName: \"kubernetes.io/projected/0ef0b8fc-02e3-44f3-9b8d-54740392b572-kube-api-access-xt6c4\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.640215 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.640118    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/0ef0b8fc-02e3-44f3-9b8d-54740392b572-metrics-client-ca\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.640366 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.640342    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-certs-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef0b8fc-02e3-44f3-9b8d-54740392b572-serving-certs-ca-bundle\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.640453 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.640431    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/0ef0b8fc-02e3-44f3-9b8d-54740392b572-telemeter-trusted-ca-bundle\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.642030 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.642007    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"federate-client-tls\" (UniqueName: \"kubernetes.io/secret/0ef0b8fc-02e3-44f3-9b8d-54740392b572-federate-client-tls\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.642121 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.642104    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client\" (UniqueName: \"kubernetes.io/secret/0ef0b8fc-02e3-44f3-9b8d-54740392b572-secret-telemeter-client\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.642414 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.642391    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-telemeter-client-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/0ef0b8fc-02e3-44f3-9b8d-54740392b572-secret-telemeter-client-kube-rbac-proxy-config\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.642470 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.642401    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"telemeter-client-tls\" (UniqueName: \"kubernetes.io/secret/0ef0b8fc-02e3-44f3-9b8d-54740392b572-telemeter-client-tls\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.646917 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.646879    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-xt6c4\" (UniqueName: \"kubernetes.io/projected/0ef0b8fc-02e3-44f3-9b8d-54740392b572-kube-api-access-xt6c4\") pod \"telemeter-client-7bbff8d7c5-2hkhw\" (UID: \"0ef0b8fc-02e3-44f3-9b8d-54740392b572\") " pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.761947 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.761842    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"
May 26 02:39:55.893327 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.893234    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw"]
May 26 02:39:55.896534 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:39:55.896506    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod0ef0b8fc_02e3_44f3_9b8d_54740392b572.slice/crio-6f998c265e0f61485f758194e2cd55fb7d9b3263b8eb51860a179992af9b1837 WatchSource:0}: Error finding container 6f998c265e0f61485f758194e2cd55fb7d9b3263b8eb51860a179992af9b1837: Status 404 returned error can't find the container with id 6f998c265e0f61485f758194e2cd55fb7d9b3263b8eb51860a179992af9b1837
May 26 02:39:55.919626 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:55.919597    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw" event={"ID":"0ef0b8fc-02e3-44f3-9b8d-54740392b572","Type":"ContainerStarted","Data":"6f998c265e0f61485f758194e2cd55fb7d9b3263b8eb51860a179992af9b1837"}
May 26 02:39:57.927371 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:57.927332    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw" event={"ID":"0ef0b8fc-02e3-44f3-9b8d-54740392b572","Type":"ContainerStarted","Data":"39c27c497a10d4c22fab324de976e080ae0d175a264e4ca98412e7c75c821752"}
May 26 02:39:58.931689 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:58.931650    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw" event={"ID":"0ef0b8fc-02e3-44f3-9b8d-54740392b572","Type":"ContainerStarted","Data":"8e60eeb691be34f7388a5d250849d915b3efd6ca1674b60829e028f7928804b7"}
May 26 02:39:58.931689 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:58.931688    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw" event={"ID":"0ef0b8fc-02e3-44f3-9b8d-54740392b572","Type":"ContainerStarted","Data":"c2212d6416deddac33e34dcd759452fe66f4d06199abda91883d23f1e856694c"}
May 26 02:39:58.957389 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:58.957342    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/telemeter-client-7bbff8d7c5-2hkhw" podStartSLOduration=1.472817037 podStartE2EDuration="3.957328928s" podCreationTimestamp="2026-05-26 02:39:55 +0000 UTC" firstStartedPulling="2026-05-26 02:39:55.898636423 +0000 UTC m=+233.455425069" lastFinishedPulling="2026-05-26 02:39:58.383148306 +0000 UTC m=+235.939936960" observedRunningTime="2026-05-26 02:39:58.955802701 +0000 UTC m=+236.512591381" watchObservedRunningTime="2026-05-26 02:39:58.957328928 +0000 UTC m=+236.514117638"
May 26 02:39:59.666953 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.666919    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-9cc8c6f98-qs2qn"]
May 26 02:39:59.669309 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.669285    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.679965 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.679938    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9cc8c6f98-qs2qn"]
May 26 02:39:59.778055 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.778020    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-trusted-ca-bundle\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.778055 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.778060    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-serving-cert\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.778256 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.778112    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-oauth-config\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.778256 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.778183    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5fh5l\" (UniqueName: \"kubernetes.io/projected/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-kube-api-access-5fh5l\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.778256 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.778207    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-service-ca\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.778256 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.778233    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-oauth-serving-cert\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.778380 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.778265    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-config\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.879513 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.879474    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5fh5l\" (UniqueName: \"kubernetes.io/projected/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-kube-api-access-5fh5l\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.879513 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.879520    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-service-ca\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.879754 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.879545    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-oauth-serving-cert\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.879754 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.879565    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-config\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.879754 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.879594    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-trusted-ca-bundle\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.879754 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.879612    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-serving-cert\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.879754 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.879636    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-oauth-config\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.880367 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.880335    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-service-ca\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.880480 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.880352    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-oauth-serving-cert\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.880480 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.880411    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-config\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.880593 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.880505    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-trusted-ca-bundle\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.882083 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.882059    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-oauth-config\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.882211 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.882192    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-serving-cert\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.887943 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.887925    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5fh5l\" (UniqueName: \"kubernetes.io/projected/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-kube-api-access-5fh5l\") pod \"console-9cc8c6f98-qs2qn\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") " pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:39:59.979805 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:39:59.979719    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:40:00.124995 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:00.124865    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-9cc8c6f98-qs2qn"]
May 26 02:40:00.127228 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:40:00.127199    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod6c6a7196_dea9_4b2e_a644_eaab470eb3aa.slice/crio-b1949feb1f5aba1fb1ede65611a9937818a8a750c9c61341f8210b42b9fe488a WatchSource:0}: Error finding container b1949feb1f5aba1fb1ede65611a9937818a8a750c9c61341f8210b42b9fe488a: Status 404 returned error can't find the container with id b1949feb1f5aba1fb1ede65611a9937818a8a750c9c61341f8210b42b9fe488a
May 26 02:40:00.939515 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:00.939478    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9cc8c6f98-qs2qn" event={"ID":"6c6a7196-dea9-4b2e-a644-eaab470eb3aa","Type":"ContainerStarted","Data":"915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0"}
May 26 02:40:00.939515 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:00.939512    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9cc8c6f98-qs2qn" event={"ID":"6c6a7196-dea9-4b2e-a644-eaab470eb3aa","Type":"ContainerStarted","Data":"b1949feb1f5aba1fb1ede65611a9937818a8a750c9c61341f8210b42b9fe488a"}
May 26 02:40:00.963737 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:00.963677    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-9cc8c6f98-qs2qn" podStartSLOduration=1.963661907 podStartE2EDuration="1.963661907s" podCreationTimestamp="2026-05-26 02:39:59 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:40:00.961532467 +0000 UTC m=+238.518321137" watchObservedRunningTime="2026-05-26 02:40:00.963661907 +0000 UTC m=+238.520450575"
May 26 02:40:09.980772 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:09.980741    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:40:09.981169 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:09.980825    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:40:09.985203 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:09.985185    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:40:10.975199 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:10.975173    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:40:11.019787 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:11.019755    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-665b4ffd55-z5xl8"]
May 26 02:40:36.040308 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.040246    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-665b4ffd55-z5xl8" podUID="d5cffbd0-be2d-4756-a4b2-c07ce183990d" containerName="console" containerID="cri-o://a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f" gracePeriod=15
May 26 02:40:36.285579 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.285556    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-665b4ffd55-z5xl8_d5cffbd0-be2d-4756-a4b2-c07ce183990d/console/0.log"
May 26 02:40:36.285704 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.285615    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:40:36.393439 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.393403    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-config\") pod \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") "
May 26 02:40:36.393439 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.393443    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-trusted-ca-bundle\") pod \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") "
May 26 02:40:36.393703 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.393475    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-service-ca\") pod \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") "
May 26 02:40:36.393703 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.393515    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-oauth-config\") pod \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") "
May 26 02:40:36.393703 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.393550    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-serving-cert\") pod \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") "
May 26 02:40:36.393703 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.393593    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-oauth-serving-cert\") pod \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") "
May 26 02:40:36.393703 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.393617    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-msspb\" (UniqueName: \"kubernetes.io/projected/d5cffbd0-be2d-4756-a4b2-c07ce183990d-kube-api-access-msspb\") pod \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\" (UID: \"d5cffbd0-be2d-4756-a4b2-c07ce183990d\") "
May 26 02:40:36.393992 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.393856    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-service-ca" (OuterVolumeSpecName: "service-ca") pod "d5cffbd0-be2d-4756-a4b2-c07ce183990d" (UID: "d5cffbd0-be2d-4756-a4b2-c07ce183990d"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:40:36.394083 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.394056    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d5cffbd0-be2d-4756-a4b2-c07ce183990d" (UID: "d5cffbd0-be2d-4756-a4b2-c07ce183990d"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:40:36.394137 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.394079    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-config" (OuterVolumeSpecName: "console-config") pod "d5cffbd0-be2d-4756-a4b2-c07ce183990d" (UID: "d5cffbd0-be2d-4756-a4b2-c07ce183990d"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:40:36.394137 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.394074    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d5cffbd0-be2d-4756-a4b2-c07ce183990d" (UID: "d5cffbd0-be2d-4756-a4b2-c07ce183990d"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:40:36.396020 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.395983    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d5cffbd0-be2d-4756-a4b2-c07ce183990d" (UID: "d5cffbd0-be2d-4756-a4b2-c07ce183990d"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:40:36.396115 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.396058    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d5cffbd0-be2d-4756-a4b2-c07ce183990d" (UID: "d5cffbd0-be2d-4756-a4b2-c07ce183990d"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:40:36.396115 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.396063    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d5cffbd0-be2d-4756-a4b2-c07ce183990d-kube-api-access-msspb" (OuterVolumeSpecName: "kube-api-access-msspb") pod "d5cffbd0-be2d-4756-a4b2-c07ce183990d" (UID: "d5cffbd0-be2d-4756-a4b2-c07ce183990d"). InnerVolumeSpecName "kube-api-access-msspb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:40:36.494468 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.494428    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-oauth-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:40:36.494468 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.494460    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-msspb\" (UniqueName: \"kubernetes.io/projected/d5cffbd0-be2d-4756-a4b2-c07ce183990d-kube-api-access-msspb\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:40:36.494468 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.494470    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:40:36.494468 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.494480    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-trusted-ca-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:40:36.494727 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.494489    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d5cffbd0-be2d-4756-a4b2-c07ce183990d-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:40:36.494727 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.494498    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-oauth-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:40:36.494727 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:36.494506    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d5cffbd0-be2d-4756-a4b2-c07ce183990d-console-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:40:37.042788 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:37.042758    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-665b4ffd55-z5xl8_d5cffbd0-be2d-4756-a4b2-c07ce183990d/console/0.log"
May 26 02:40:37.043175 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:37.042799    2551 generic.go:358] "Generic (PLEG): container finished" podID="d5cffbd0-be2d-4756-a4b2-c07ce183990d" containerID="a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f" exitCode=2
May 26 02:40:37.043175 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:37.042910    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-665b4ffd55-z5xl8"
May 26 02:40:37.044007 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:37.043980    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-665b4ffd55-z5xl8" event={"ID":"d5cffbd0-be2d-4756-a4b2-c07ce183990d","Type":"ContainerDied","Data":"a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f"}
May 26 02:40:37.044133 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:37.044012    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-665b4ffd55-z5xl8" event={"ID":"d5cffbd0-be2d-4756-a4b2-c07ce183990d","Type":"ContainerDied","Data":"45dece4b3049684d31b9633f3dbd73113b36a4c4ec1e53b7786cd3cb35d9e65b"}
May 26 02:40:37.044133 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:37.044032    2551 scope.go:117] "RemoveContainer" containerID="a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f"
May 26 02:40:37.052104 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:37.052087    2551 scope.go:117] "RemoveContainer" containerID="a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f"
May 26 02:40:37.052408 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:40:37.052383    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f\": container with ID starting with a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f not found: ID does not exist" containerID="a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f"
May 26 02:40:37.052445 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:37.052421    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f"} err="failed to get container status \"a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f\": rpc error: code = NotFound desc = could not find container \"a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f\": container with ID starting with a2d7cf73e58868ee690d3a79c8170a844675d472fd4b635a3a01e945df50ba6f not found: ID does not exist"
May 26 02:40:37.065683 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:37.065644    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-665b4ffd55-z5xl8"]
May 26 02:40:37.069722 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:37.069694    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-665b4ffd55-z5xl8"]
May 26 02:40:39.044262 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:40:39.044229    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d5cffbd0-be2d-4756-a4b2-c07ce183990d" path="/var/lib/kubelet/pods/d5cffbd0-be2d-4756-a4b2-c07ce183990d/volumes"
May 26 02:41:02.916653 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:02.916610    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 02:41:02.918087 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:02.918061    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 02:41:02.922461 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:02.922432    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 02:41:02.923535 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:02.923510    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 02:41:02.927776 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:02.927756    2551 kubelet.go:1628] "Image garbage collection succeeded"
May 26 02:41:11.958271 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:11.958230    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-64b996cd7-7kkqm"]
May 26 02:41:11.960842 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:11.958607    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d5cffbd0-be2d-4756-a4b2-c07ce183990d" containerName="console"
May 26 02:41:11.960842 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:11.958620    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d5cffbd0-be2d-4756-a4b2-c07ce183990d" containerName="console"
May 26 02:41:11.960842 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:11.958669    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="d5cffbd0-be2d-4756-a4b2-c07ce183990d" containerName="console"
May 26 02:41:11.961749 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:11.961725    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:11.970222 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:11.970195    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64b996cd7-7kkqm"]
May 26 02:41:12.088780 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.088746    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d947c490-d87e-4291-9891-1dcde8247601-console-oauth-config\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.088981 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.088803    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-oauth-serving-cert\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.088981 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.088885    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-service-ca\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.088981 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.088956    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d947c490-d87e-4291-9891-1dcde8247601-console-serving-cert\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.088981 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.088979    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-trusted-ca-bundle\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.089137 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.089004    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-nw2k2\" (UniqueName: \"kubernetes.io/projected/d947c490-d87e-4291-9891-1dcde8247601-kube-api-access-nw2k2\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.089137 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.089025    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-console-config\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.190128 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.190089    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-oauth-serving-cert\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.190128 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.190136    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-service-ca\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.190380 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.190177    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d947c490-d87e-4291-9891-1dcde8247601-console-serving-cert\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.190380 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.190203    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-trusted-ca-bundle\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.190380 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.190234    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-nw2k2\" (UniqueName: \"kubernetes.io/projected/d947c490-d87e-4291-9891-1dcde8247601-kube-api-access-nw2k2\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.190380 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.190262    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-console-config\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.190380 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.190296    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d947c490-d87e-4291-9891-1dcde8247601-console-oauth-config\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.191004 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.190976    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-oauth-serving-cert\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.191125 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.191005    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-console-config\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.191125 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.191007    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-service-ca\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.191125 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.191109    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-trusted-ca-bundle\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.192693 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.192667    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d947c490-d87e-4291-9891-1dcde8247601-console-oauth-config\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.192784 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.192763    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d947c490-d87e-4291-9891-1dcde8247601-console-serving-cert\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.197500 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.197480    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-nw2k2\" (UniqueName: \"kubernetes.io/projected/d947c490-d87e-4291-9891-1dcde8247601-kube-api-access-nw2k2\") pod \"console-64b996cd7-7kkqm\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") " pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.273313 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.273229    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:12.400193 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.400167    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-64b996cd7-7kkqm"]
May 26 02:41:12.402501 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:41:12.402465    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd947c490_d87e_4291_9891_1dcde8247601.slice/crio-2cde5223c16daee6fe80faf4dfae8db8be29c3f547afe9f27889546e8e60b161 WatchSource:0}: Error finding container 2cde5223c16daee6fe80faf4dfae8db8be29c3f547afe9f27889546e8e60b161: Status 404 returned error can't find the container with id 2cde5223c16daee6fe80faf4dfae8db8be29c3f547afe9f27889546e8e60b161
May 26 02:41:12.404284 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:12.404264    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 26 02:41:13.144910 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:13.144874    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64b996cd7-7kkqm" event={"ID":"d947c490-d87e-4291-9891-1dcde8247601","Type":"ContainerStarted","Data":"ec1c15d59217f2ea851dc4b904c4d74f6a936884a41b466772082076d15ec322"}
May 26 02:41:13.145309 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:13.144923    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64b996cd7-7kkqm" event={"ID":"d947c490-d87e-4291-9891-1dcde8247601","Type":"ContainerStarted","Data":"2cde5223c16daee6fe80faf4dfae8db8be29c3f547afe9f27889546e8e60b161"}
May 26 02:41:13.161620 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:13.161573    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-64b996cd7-7kkqm" podStartSLOduration=2.161559226 podStartE2EDuration="2.161559226s" podCreationTimestamp="2026-05-26 02:41:11 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:41:13.160037235 +0000 UTC m=+310.716825903" watchObservedRunningTime="2026-05-26 02:41:13.161559226 +0000 UTC m=+310.718347894"
May 26 02:41:22.274030 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:22.273974    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:22.274030 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:22.274035    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:22.278579 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:22.278557    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:23.174683 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:23.174656    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:41:23.218628 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:23.218586    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9cc8c6f98-qs2qn"]
May 26 02:41:44.700711 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.700677    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-kj9cp"]
May 26 02:41:44.704080 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.704062    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj9cp"
May 26 02:41:44.706325 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.706302    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 26 02:41:44.711190 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.711168    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kj9cp"]
May 26 02:41:44.877882 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.877829    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995958a8-97ad-4e57-96be-498290f87329-original-pull-secret\") pod \"global-pull-secret-syncer-kj9cp\" (UID: \"995958a8-97ad-4e57-96be-498290f87329\") " pod="kube-system/global-pull-secret-syncer-kj9cp"
May 26 02:41:44.877882 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.877894    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/995958a8-97ad-4e57-96be-498290f87329-dbus\") pod \"global-pull-secret-syncer-kj9cp\" (UID: \"995958a8-97ad-4e57-96be-498290f87329\") " pod="kube-system/global-pull-secret-syncer-kj9cp"
May 26 02:41:44.878128 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.877981    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/995958a8-97ad-4e57-96be-498290f87329-kubelet-config\") pod \"global-pull-secret-syncer-kj9cp\" (UID: \"995958a8-97ad-4e57-96be-498290f87329\") " pod="kube-system/global-pull-secret-syncer-kj9cp"
May 26 02:41:44.978984 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.978865    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995958a8-97ad-4e57-96be-498290f87329-original-pull-secret\") pod \"global-pull-secret-syncer-kj9cp\" (UID: \"995958a8-97ad-4e57-96be-498290f87329\") " pod="kube-system/global-pull-secret-syncer-kj9cp"
May 26 02:41:44.978984 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.978943    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/995958a8-97ad-4e57-96be-498290f87329-dbus\") pod \"global-pull-secret-syncer-kj9cp\" (UID: \"995958a8-97ad-4e57-96be-498290f87329\") " pod="kube-system/global-pull-secret-syncer-kj9cp"
May 26 02:41:44.979190 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.978991    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/995958a8-97ad-4e57-96be-498290f87329-kubelet-config\") pod \"global-pull-secret-syncer-kj9cp\" (UID: \"995958a8-97ad-4e57-96be-498290f87329\") " pod="kube-system/global-pull-secret-syncer-kj9cp"
May 26 02:41:44.979190 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.979072    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/995958a8-97ad-4e57-96be-498290f87329-kubelet-config\") pod \"global-pull-secret-syncer-kj9cp\" (UID: \"995958a8-97ad-4e57-96be-498290f87329\") " pod="kube-system/global-pull-secret-syncer-kj9cp"
May 26 02:41:44.979190 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.979152    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/995958a8-97ad-4e57-96be-498290f87329-dbus\") pod \"global-pull-secret-syncer-kj9cp\" (UID: \"995958a8-97ad-4e57-96be-498290f87329\") " pod="kube-system/global-pull-secret-syncer-kj9cp"
May 26 02:41:44.981173 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:44.981156    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/995958a8-97ad-4e57-96be-498290f87329-original-pull-secret\") pod \"global-pull-secret-syncer-kj9cp\" (UID: \"995958a8-97ad-4e57-96be-498290f87329\") " pod="kube-system/global-pull-secret-syncer-kj9cp"
May 26 02:41:45.014186 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:45.014151    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-kj9cp"
May 26 02:41:45.137058 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:45.137005    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-kj9cp"]
May 26 02:41:45.139334 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:41:45.139309    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod995958a8_97ad_4e57_96be_498290f87329.slice/crio-77ef0563feef32d69ad2830df391767f7dac7e6e9cc54d1fb3e898df88f4103d WatchSource:0}: Error finding container 77ef0563feef32d69ad2830df391767f7dac7e6e9cc54d1fb3e898df88f4103d: Status 404 returned error can't find the container with id 77ef0563feef32d69ad2830df391767f7dac7e6e9cc54d1fb3e898df88f4103d
May 26 02:41:45.237209 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:45.237106    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kj9cp" event={"ID":"995958a8-97ad-4e57-96be-498290f87329","Type":"ContainerStarted","Data":"77ef0563feef32d69ad2830df391767f7dac7e6e9cc54d1fb3e898df88f4103d"}
May 26 02:41:48.238580 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:48.238523    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-9cc8c6f98-qs2qn" podUID="6c6a7196-dea9-4b2e-a644-eaab470eb3aa" containerName="console" containerID="cri-o://915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0" gracePeriod=15
May 26 02:41:49.007321 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.007297    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-9cc8c6f98-qs2qn_6c6a7196-dea9-4b2e-a644-eaab470eb3aa/console/0.log"
May 26 02:41:49.007421 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.007357    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:41:49.114164 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.114076    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-trusted-ca-bundle\") pod \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") "
May 26 02:41:49.114164 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.114115    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-oauth-config\") pod \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") "
May 26 02:41:49.114368 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.114179    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-serving-cert\") pod \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") "
May 26 02:41:49.114368 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.114221    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-config\") pod \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") "
May 26 02:41:49.114368 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.114303    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-service-ca\") pod \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") "
May 26 02:41:49.114368 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.114331    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-5fh5l\" (UniqueName: \"kubernetes.io/projected/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-kube-api-access-5fh5l\") pod \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") "
May 26 02:41:49.114368 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.114363    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-oauth-serving-cert\") pod \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\" (UID: \"6c6a7196-dea9-4b2e-a644-eaab470eb3aa\") "
May 26 02:41:49.114599 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.114488    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "6c6a7196-dea9-4b2e-a644-eaab470eb3aa" (UID: "6c6a7196-dea9-4b2e-a644-eaab470eb3aa"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:41:49.114719 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.114701    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-trusted-ca-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:41:49.114800 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.114720    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-config" (OuterVolumeSpecName: "console-config") pod "6c6a7196-dea9-4b2e-a644-eaab470eb3aa" (UID: "6c6a7196-dea9-4b2e-a644-eaab470eb3aa"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:41:49.114800 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.114742    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "6c6a7196-dea9-4b2e-a644-eaab470eb3aa" (UID: "6c6a7196-dea9-4b2e-a644-eaab470eb3aa"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:41:49.115367 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.115339    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-service-ca" (OuterVolumeSpecName: "service-ca") pod "6c6a7196-dea9-4b2e-a644-eaab470eb3aa" (UID: "6c6a7196-dea9-4b2e-a644-eaab470eb3aa"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:41:49.116759 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.116730    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "6c6a7196-dea9-4b2e-a644-eaab470eb3aa" (UID: "6c6a7196-dea9-4b2e-a644-eaab470eb3aa"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:41:49.116968 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.116950    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "6c6a7196-dea9-4b2e-a644-eaab470eb3aa" (UID: "6c6a7196-dea9-4b2e-a644-eaab470eb3aa"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:41:49.117033 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.117012    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-kube-api-access-5fh5l" (OuterVolumeSpecName: "kube-api-access-5fh5l") pod "6c6a7196-dea9-4b2e-a644-eaab470eb3aa" (UID: "6c6a7196-dea9-4b2e-a644-eaab470eb3aa"). InnerVolumeSpecName "kube-api-access-5fh5l". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:41:49.215678 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.215645    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:41:49.215678 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.215672    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:41:49.215678 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.215682    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:41:49.215929 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.215691    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-5fh5l\" (UniqueName: \"kubernetes.io/projected/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-kube-api-access-5fh5l\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:41:49.215929 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.215700    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-oauth-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:41:49.215929 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.215708    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/6c6a7196-dea9-4b2e-a644-eaab470eb3aa-console-oauth-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:41:49.250590 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.250553    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-kj9cp" event={"ID":"995958a8-97ad-4e57-96be-498290f87329","Type":"ContainerStarted","Data":"111f6b14586824b76f757e4b8f9c6a546d1015cda66d857b83c8f0c18a85d822"}
May 26 02:41:49.251610 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.251591    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-9cc8c6f98-qs2qn_6c6a7196-dea9-4b2e-a644-eaab470eb3aa/console/0.log"
May 26 02:41:49.251726 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.251628    2551 generic.go:358] "Generic (PLEG): container finished" podID="6c6a7196-dea9-4b2e-a644-eaab470eb3aa" containerID="915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0" exitCode=2
May 26 02:41:49.251726 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.251676    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9cc8c6f98-qs2qn" event={"ID":"6c6a7196-dea9-4b2e-a644-eaab470eb3aa","Type":"ContainerDied","Data":"915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0"}
May 26 02:41:49.251726 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.251679    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-9cc8c6f98-qs2qn"
May 26 02:41:49.251726 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.251711    2551 scope.go:117] "RemoveContainer" containerID="915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0"
May 26 02:41:49.252010 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.251699    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-9cc8c6f98-qs2qn" event={"ID":"6c6a7196-dea9-4b2e-a644-eaab470eb3aa","Type":"ContainerDied","Data":"b1949feb1f5aba1fb1ede65611a9937818a8a750c9c61341f8210b42b9fe488a"}
May 26 02:41:49.260106 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.260090    2551 scope.go:117] "RemoveContainer" containerID="915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0"
May 26 02:41:49.260354 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:41:49.260336    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0\": container with ID starting with 915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0 not found: ID does not exist" containerID="915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0"
May 26 02:41:49.260397 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.260362    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0"} err="failed to get container status \"915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0\": rpc error: code = NotFound desc = could not find container \"915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0\": container with ID starting with 915d3e0c5e58b3e372575141b0a4e2c1abc78abfb519c1a9a7b55a5f93c118d0 not found: ID does not exist"
May 26 02:41:49.266246 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.266202    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-kj9cp" podStartSLOduration=1.513934764 podStartE2EDuration="5.266190824s" podCreationTimestamp="2026-05-26 02:41:44 +0000 UTC" firstStartedPulling="2026-05-26 02:41:45.141278727 +0000 UTC m=+342.698067389" lastFinishedPulling="2026-05-26 02:41:48.893534802 +0000 UTC m=+346.450323449" observedRunningTime="2026-05-26 02:41:49.264575668 +0000 UTC m=+346.821364340" watchObservedRunningTime="2026-05-26 02:41:49.266190824 +0000 UTC m=+346.822979492"
May 26 02:41:49.276638 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.276613    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-9cc8c6f98-qs2qn"]
May 26 02:41:49.281782 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:49.281759    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-9cc8c6f98-qs2qn"]
May 26 02:41:51.044383 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:41:51.044347    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6c6a7196-dea9-4b2e-a644-eaab470eb3aa" path="/var/lib/kubelet/pods/6c6a7196-dea9-4b2e-a644-eaab470eb3aa/volumes"
May 26 02:44:31.477144 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.477100    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"]
May 26 02:44:31.477612 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.477390    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6c6a7196-dea9-4b2e-a644-eaab470eb3aa" containerName="console"
May 26 02:44:31.477612 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.477400    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="6c6a7196-dea9-4b2e-a644-eaab470eb3aa" containerName="console"
May 26 02:44:31.477612 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.477465    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="6c6a7196-dea9-4b2e-a644-eaab470eb3aa" containerName="console"
May 26 02:44:31.479610 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.479593    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"
May 26 02:44:31.481844 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.481795    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"openshift-service-ca.crt\""
May 26 02:44:31.481978 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.481954    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"feast-operator-system\"/\"feast-operator-controller-manager-dockercfg-4stdx\""
May 26 02:44:31.482679 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.482661    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"feast-operator-system\"/\"kube-root-ca.crt\""
May 26 02:44:31.497282 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.497260    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"]
May 26 02:44:31.556619 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.556591    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-x2752\" (UniqueName: \"kubernetes.io/projected/8fa41aea-c330-460d-8a3f-118ec9449f26-kube-api-access-x2752\") pod \"feast-operator-controller-manager-5f8946788d-tzhpc\" (UID: \"8fa41aea-c330-460d-8a3f-118ec9449f26\") " pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"
May 26 02:44:31.657080 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.657044    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-x2752\" (UniqueName: \"kubernetes.io/projected/8fa41aea-c330-460d-8a3f-118ec9449f26-kube-api-access-x2752\") pod \"feast-operator-controller-manager-5f8946788d-tzhpc\" (UID: \"8fa41aea-c330-460d-8a3f-118ec9449f26\") " pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"
May 26 02:44:31.665058 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.665029    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-x2752\" (UniqueName: \"kubernetes.io/projected/8fa41aea-c330-460d-8a3f-118ec9449f26-kube-api-access-x2752\") pod \"feast-operator-controller-manager-5f8946788d-tzhpc\" (UID: \"8fa41aea-c330-460d-8a3f-118ec9449f26\") " pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"
May 26 02:44:31.789951 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.789861    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"
May 26 02:44:31.908469 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:31.908436    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"]
May 26 02:44:31.911314 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:44:31.911281    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod8fa41aea_c330_460d_8a3f_118ec9449f26.slice/crio-0406e6f41420d83ec99fd99704e01d09cedbf95fc730234163041176957a37bd WatchSource:0}: Error finding container 0406e6f41420d83ec99fd99704e01d09cedbf95fc730234163041176957a37bd: Status 404 returned error can't find the container with id 0406e6f41420d83ec99fd99704e01d09cedbf95fc730234163041176957a37bd
May 26 02:44:32.716994 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:32.716954    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc" event={"ID":"8fa41aea-c330-460d-8a3f-118ec9449f26","Type":"ContainerStarted","Data":"0406e6f41420d83ec99fd99704e01d09cedbf95fc730234163041176957a37bd"}
May 26 02:44:34.725244 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:34.725154    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc" event={"ID":"8fa41aea-c330-460d-8a3f-118ec9449f26","Type":"ContainerStarted","Data":"7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2"}
May 26 02:44:34.725677 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:34.725278    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"
May 26 02:44:34.740315 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:34.740254    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc" podStartSLOduration=1.300101368 podStartE2EDuration="3.740237907s" podCreationTimestamp="2026-05-26 02:44:31 +0000 UTC" firstStartedPulling="2026-05-26 02:44:31.913145749 +0000 UTC m=+509.469934400" lastFinishedPulling="2026-05-26 02:44:34.353282277 +0000 UTC m=+511.910070939" observedRunningTime="2026-05-26 02:44:34.739804864 +0000 UTC m=+512.296593535" watchObservedRunningTime="2026-05-26 02:44:34.740237907 +0000 UTC m=+512.297026582"
May 26 02:44:45.730988 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:44:45.730915    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"
May 26 02:45:02.857546 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:02.857493    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-74f8df89f5-rhbtk"]
May 26 02:45:02.864354 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:02.864330    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:02.871964 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:02.871600    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74f8df89f5-rhbtk"]
May 26 02:45:02.998008 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:02.997973    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9d8feaf-5d5b-41c9-a635-f475a317ce13-console-oauth-config\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:02.998008 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:02.998013    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9d8feaf-5d5b-41c9-a635-f475a317ce13-console-serving-cert\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:02.998194 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:02.998031    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9d8feaf-5d5b-41c9-a635-f475a317ce13-console-config\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:02.998194 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:02.998116    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d8feaf-5d5b-41c9-a635-f475a317ce13-service-ca\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:02.998194 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:02.998158    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9d8feaf-5d5b-41c9-a635-f475a317ce13-oauth-serving-cert\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:02.998194 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:02.998184    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9d8feaf-5d5b-41c9-a635-f475a317ce13-trusted-ca-bundle\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:02.998317 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:02.998214    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2nsnz\" (UniqueName: \"kubernetes.io/projected/f9d8feaf-5d5b-41c9-a635-f475a317ce13-kube-api-access-2nsnz\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.098985 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.098942    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9d8feaf-5d5b-41c9-a635-f475a317ce13-console-oauth-config\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.098985 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.098993    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9d8feaf-5d5b-41c9-a635-f475a317ce13-console-serving-cert\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.099227 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.099011    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9d8feaf-5d5b-41c9-a635-f475a317ce13-console-config\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.099227 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.099044    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d8feaf-5d5b-41c9-a635-f475a317ce13-service-ca\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.099227 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.099073    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9d8feaf-5d5b-41c9-a635-f475a317ce13-oauth-serving-cert\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.099227 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.099094    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9d8feaf-5d5b-41c9-a635-f475a317ce13-trusted-ca-bundle\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.099227 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.099198    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2nsnz\" (UniqueName: \"kubernetes.io/projected/f9d8feaf-5d5b-41c9-a635-f475a317ce13-kube-api-access-2nsnz\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.099850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.099820    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/f9d8feaf-5d5b-41c9-a635-f475a317ce13-console-config\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.099990 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.099836    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/f9d8feaf-5d5b-41c9-a635-f475a317ce13-service-ca\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.099990 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.099923    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/f9d8feaf-5d5b-41c9-a635-f475a317ce13-trusted-ca-bundle\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.099990 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.099945    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/f9d8feaf-5d5b-41c9-a635-f475a317ce13-oauth-serving-cert\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.101373 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.101349    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/f9d8feaf-5d5b-41c9-a635-f475a317ce13-console-serving-cert\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.101472 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.101379    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/f9d8feaf-5d5b-41c9-a635-f475a317ce13-console-oauth-config\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.106277 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.106256    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2nsnz\" (UniqueName: \"kubernetes.io/projected/f9d8feaf-5d5b-41c9-a635-f475a317ce13-kube-api-access-2nsnz\") pod \"console-74f8df89f5-rhbtk\" (UID: \"f9d8feaf-5d5b-41c9-a635-f475a317ce13\") " pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.175217 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.175165    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:03.300739 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.300659    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-74f8df89f5-rhbtk"]
May 26 02:45:03.303159 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:45:03.303136    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podf9d8feaf_5d5b_41c9_a635_f475a317ce13.slice/crio-3aaed0c93f74c4cb29f5707dcd25bfff02e2efa49f623b8af4560950c8ef9d50 WatchSource:0}: Error finding container 3aaed0c93f74c4cb29f5707dcd25bfff02e2efa49f623b8af4560950c8ef9d50: Status 404 returned error can't find the container with id 3aaed0c93f74c4cb29f5707dcd25bfff02e2efa49f623b8af4560950c8ef9d50
May 26 02:45:03.814344 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.814308    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f8df89f5-rhbtk" event={"ID":"f9d8feaf-5d5b-41c9-a635-f475a317ce13","Type":"ContainerStarted","Data":"5942b8cff7495cc3f25a88af288a5c19541922ea2b292665bb7d19042c1b1783"}
May 26 02:45:03.814344 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.814347    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-74f8df89f5-rhbtk" event={"ID":"f9d8feaf-5d5b-41c9-a635-f475a317ce13","Type":"ContainerStarted","Data":"3aaed0c93f74c4cb29f5707dcd25bfff02e2efa49f623b8af4560950c8ef9d50"}
May 26 02:45:03.829957 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:03.829877    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-74f8df89f5-rhbtk" podStartSLOduration=1.829860734 podStartE2EDuration="1.829860734s" podCreationTimestamp="2026-05-26 02:45:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:45:03.828539237 +0000 UTC m=+541.385327904" watchObservedRunningTime="2026-05-26 02:45:03.829860734 +0000 UTC m=+541.386649405"
May 26 02:45:13.175486 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:13.175451    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:13.175486 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:13.175492    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:13.179938 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:13.179918    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:13.857690 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:13.857659    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-74f8df89f5-rhbtk"
May 26 02:45:13.901462 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:13.901429    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64b996cd7-7kkqm"]
May 26 02:45:13.991992 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:13.991959    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"]
May 26 02:45:13.996232 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:13.996211    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:13.999174 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:13.999152    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-xqhdj\""
May 26 02:45:13.999631 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:13.999614    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 26 02:45:13.999961 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:13.999945    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 26 02:45:14.000248 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.000231    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 26 02:45:14.000620 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.000603    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 26 02:45:14.001046 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.001029    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 26 02:45:14.001561 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.001462    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 26 02:45:14.001885 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.001866    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 26 02:45:14.011730 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.011679    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"]
May 26 02:45:14.089349 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.089317    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jhpq8\" (UniqueName: \"kubernetes.io/projected/99af60ce-315a-481c-a29e-9c8e7e9de804-kube-api-access-jhpq8\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.089349 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.089362    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-online-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.089573 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.089398    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-offline-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.089573 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.089443    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/99af60ce-315a-481c-a29e-9c8e7e9de804-feast-data\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.089573 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.089491    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-ui-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.089573 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.089527    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-registry-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.189948 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.189855    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-registry-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.190383 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.189973    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jhpq8\" (UniqueName: \"kubernetes.io/projected/99af60ce-315a-481c-a29e-9c8e7e9de804-kube-api-access-jhpq8\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.190383 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.190004    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-online-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.190383 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.190040    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-offline-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.190383 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.190067    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/99af60ce-315a-481c-a29e-9c8e7e9de804-feast-data\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.190383 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.190219    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-ui-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.190659 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.190403    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/99af60ce-315a-481c-a29e-9c8e7e9de804-feast-data\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.192460 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.192428    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-online-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.192594 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.192506    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-offline-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.192701 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.192680    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-registry-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.192804 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.192786    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-ui-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.197450 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.197418    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jhpq8\" (UniqueName: \"kubernetes.io/projected/99af60ce-315a-481c-a29e-9c8e7e9de804-kube-api-access-jhpq8\") pod \"feast-simple-feast-setup-8bf5d45fd-555g9\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.321005 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.320967    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:14.454220 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.454192    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"]
May 26 02:45:14.456989 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:45:14.456958    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod99af60ce_315a_481c_a29e_9c8e7e9de804.slice/crio-b088f1c34b3b2a3d550e84b4b27bc138534d34c942b54f87a2abd2dc7b4fdcb4 WatchSource:0}: Error finding container b088f1c34b3b2a3d550e84b4b27bc138534d34c942b54f87a2abd2dc7b4fdcb4: Status 404 returned error can't find the container with id b088f1c34b3b2a3d550e84b4b27bc138534d34c942b54f87a2abd2dc7b4fdcb4
May 26 02:45:14.850886 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:14.850854    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerStarted","Data":"b088f1c34b3b2a3d550e84b4b27bc138534d34c942b54f87a2abd2dc7b4fdcb4"}
May 26 02:45:38.930612 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:38.930547    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-64b996cd7-7kkqm" podUID="d947c490-d87e-4291-9891-1dcde8247601" containerName="console" containerID="cri-o://ec1c15d59217f2ea851dc4b904c4d74f6a936884a41b466772082076d15ec322" gracePeriod=15
May 26 02:45:43.172182 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:43.172149    2551 patch_prober.go:28] interesting pod/console-64b996cd7-7kkqm container/console namespace/openshift-console: Readiness probe status=failure output="Get \"https://10.134.0.24:8443/health\": dial tcp 10.134.0.24:8443: connect: connection refused" start-of-body=
May 26 02:45:43.172652 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:43.172222    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console/console-64b996cd7-7kkqm" podUID="d947c490-d87e-4291-9891-1dcde8247601" containerName="console" probeResult="failure" output="Get \"https://10.134.0.24:8443/health\": dial tcp 10.134.0.24:8443: connect: connection refused"
May 26 02:45:43.946778 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:43.946750    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64b996cd7-7kkqm_d947c490-d87e-4291-9891-1dcde8247601/console/0.log"
May 26 02:45:43.946948 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:43.946793    2551 generic.go:358] "Generic (PLEG): container finished" podID="d947c490-d87e-4291-9891-1dcde8247601" containerID="ec1c15d59217f2ea851dc4b904c4d74f6a936884a41b466772082076d15ec322" exitCode=2
May 26 02:45:43.946948 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:43.946878    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64b996cd7-7kkqm" event={"ID":"d947c490-d87e-4291-9891-1dcde8247601","Type":"ContainerDied","Data":"ec1c15d59217f2ea851dc4b904c4d74f6a936884a41b466772082076d15ec322"}
May 26 02:45:44.042175 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.042156    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64b996cd7-7kkqm_d947c490-d87e-4291-9891-1dcde8247601/console/0.log"
May 26 02:45:44.042272 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.042212    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:45:44.165935 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.165822    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-console-config\") pod \"d947c490-d87e-4291-9891-1dcde8247601\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") "
May 26 02:45:44.165935 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.165873    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-nw2k2\" (UniqueName: \"kubernetes.io/projected/d947c490-d87e-4291-9891-1dcde8247601-kube-api-access-nw2k2\") pod \"d947c490-d87e-4291-9891-1dcde8247601\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") "
May 26 02:45:44.165935 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.165930    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d947c490-d87e-4291-9891-1dcde8247601-console-oauth-config\") pod \"d947c490-d87e-4291-9891-1dcde8247601\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") "
May 26 02:45:44.166206 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.165950    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-oauth-serving-cert\") pod \"d947c490-d87e-4291-9891-1dcde8247601\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") "
May 26 02:45:44.166206 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.165981    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-service-ca\") pod \"d947c490-d87e-4291-9891-1dcde8247601\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") "
May 26 02:45:44.166206 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.166152    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d947c490-d87e-4291-9891-1dcde8247601-console-serving-cert\") pod \"d947c490-d87e-4291-9891-1dcde8247601\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") "
May 26 02:45:44.166374 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.166218    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-trusted-ca-bundle\") pod \"d947c490-d87e-4291-9891-1dcde8247601\" (UID: \"d947c490-d87e-4291-9891-1dcde8247601\") "
May 26 02:45:44.166374 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.166311    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-console-config" (OuterVolumeSpecName: "console-config") pod "d947c490-d87e-4291-9891-1dcde8247601" (UID: "d947c490-d87e-4291-9891-1dcde8247601"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:45:44.166374 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.166348    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-service-ca" (OuterVolumeSpecName: "service-ca") pod "d947c490-d87e-4291-9891-1dcde8247601" (UID: "d947c490-d87e-4291-9891-1dcde8247601"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:45:44.166596 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.166574    2551 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-service-ca\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:45:44.166720 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.166599    2551 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-console-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:45:44.166720 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.166616    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "d947c490-d87e-4291-9891-1dcde8247601" (UID: "d947c490-d87e-4291-9891-1dcde8247601"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:45:44.166720 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.166637    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "d947c490-d87e-4291-9891-1dcde8247601" (UID: "d947c490-d87e-4291-9891-1dcde8247601"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:45:44.168198 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.168172    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d947c490-d87e-4291-9891-1dcde8247601-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "d947c490-d87e-4291-9891-1dcde8247601" (UID: "d947c490-d87e-4291-9891-1dcde8247601"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:45:44.168198 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.168183    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d947c490-d87e-4291-9891-1dcde8247601-kube-api-access-nw2k2" (OuterVolumeSpecName: "kube-api-access-nw2k2") pod "d947c490-d87e-4291-9891-1dcde8247601" (UID: "d947c490-d87e-4291-9891-1dcde8247601"). InnerVolumeSpecName "kube-api-access-nw2k2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:45:44.168384 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.168363    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d947c490-d87e-4291-9891-1dcde8247601-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "d947c490-d87e-4291-9891-1dcde8247601" (UID: "d947c490-d87e-4291-9891-1dcde8247601"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:45:44.270187 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.270090    2551 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/d947c490-d87e-4291-9891-1dcde8247601-console-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:45:44.270187 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.270119    2551 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-trusted-ca-bundle\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:45:44.270187 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.270134    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-nw2k2\" (UniqueName: \"kubernetes.io/projected/d947c490-d87e-4291-9891-1dcde8247601-kube-api-access-nw2k2\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:45:44.270187 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.270150    2551 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/d947c490-d87e-4291-9891-1dcde8247601-console-oauth-config\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:45:44.270187 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.270164    2551 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/d947c490-d87e-4291-9891-1dcde8247601-oauth-serving-cert\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:45:44.951334 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.951308    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-64b996cd7-7kkqm_d947c490-d87e-4291-9891-1dcde8247601/console/0.log"
May 26 02:45:44.951547 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.951426    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-64b996cd7-7kkqm"
May 26 02:45:44.951547 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.951443    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-64b996cd7-7kkqm" event={"ID":"d947c490-d87e-4291-9891-1dcde8247601","Type":"ContainerDied","Data":"2cde5223c16daee6fe80faf4dfae8db8be29c3f547afe9f27889546e8e60b161"}
May 26 02:45:44.951547 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.951499    2551 scope.go:117] "RemoveContainer" containerID="ec1c15d59217f2ea851dc4b904c4d74f6a936884a41b466772082076d15ec322"
May 26 02:45:44.952963 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.952885    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerStarted","Data":"eb1415e81fb8ae975162f96baa85b7a83aed13558d544b6cccf67da7d765488f"}
May 26 02:45:44.992372 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.992303    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-64b996cd7-7kkqm"]
May 26 02:45:44.995642 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:44.995619    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-64b996cd7-7kkqm"]
May 26 02:45:45.045893 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:45.045860    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d947c490-d87e-4291-9891-1dcde8247601" path="/var/lib/kubelet/pods/d947c490-d87e-4291-9891-1dcde8247601/volumes"
May 26 02:45:48.967486 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:48.967406    2551 generic.go:358] "Generic (PLEG): container finished" podID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerID="eb1415e81fb8ae975162f96baa85b7a83aed13558d544b6cccf67da7d765488f" exitCode=0
May 26 02:45:48.967856 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:48.967481    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerDied","Data":"eb1415e81fb8ae975162f96baa85b7a83aed13558d544b6cccf67da7d765488f"}
May 26 02:45:49.972347 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:49.972304    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerStarted","Data":"42551d10439f1d39b9cd4283318c5ad0a688fcafaf30cabf03ff61e15a7ae315"}
May 26 02:45:54.990417 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:54.990382    2551 generic.go:358] "Generic (PLEG): container finished" podID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerID="42551d10439f1d39b9cd4283318c5ad0a688fcafaf30cabf03ff61e15a7ae315" exitCode=0
May 26 02:45:54.990813 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:54.990450    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerDied","Data":"42551d10439f1d39b9cd4283318c5ad0a688fcafaf30cabf03ff61e15a7ae315"}
May 26 02:45:55.998368 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:55.998327    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerStarted","Data":"9e766ba4bf561ca6944536da94828c9e8cbefc6701dfef75e77e9f790fa3d84d"}
May 26 02:45:55.998915 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:55.998380    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerStarted","Data":"262747188eb8443138438cdd66b46b96aaf0dfb1e47a51b8ff9b78c2457c2385"}
May 26 02:45:55.998915 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:55.998398    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerStarted","Data":"f6dddf3f4bb855ddb22c4928d5a5c1bf4a0107d4e449b662f3d88615b7df8814"}
May 26 02:45:55.998915 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:55.998411    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerStarted","Data":"1bb7210141ac85ff1357192ecea058a8056315379dd12bf0c9f85c844bfbf808"}
May 26 02:45:56.021263 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:56.021201    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podStartSLOduration=13.463654127 podStartE2EDuration="43.021180401s" podCreationTimestamp="2026-05-26 02:45:13 +0000 UTC" firstStartedPulling="2026-05-26 02:45:14.458916937 +0000 UTC m=+552.015705597" lastFinishedPulling="2026-05-26 02:45:44.016443221 +0000 UTC m=+581.573231871" observedRunningTime="2026-05-26 02:45:56.020064318 +0000 UTC m=+593.576852990" watchObservedRunningTime="2026-05-26 02:45:56.021180401 +0000 UTC m=+593.577969069"
May 26 02:45:56.322086 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:56.322042    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:56.322361 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:56.322333    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:56.322361 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:56.322358    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:56.322538 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:56.322371    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:45:56.324527 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:56.324489    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 26 02:45:56.324695 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:56.324532    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 26 02:45:56.324695 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:56.324619    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused"
May 26 02:45:56.324914 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:56.324860    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused"
May 26 02:45:59.322213 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:59.322156    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 26 02:45:59.322694 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:59.322156    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused"
May 26 02:45:59.322694 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:59.322276    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 26 02:45:59.322694 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:45:59.322659    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused"
May 26 02:46:02.322047 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.322011    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:46:02.322631 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.322609    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:46:02.322745 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.322732    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:46:02.323177 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.323046    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:46:02.323177 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.323085    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:46:02.323177 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.323097    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:46:02.323405 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.323384    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:46:02.323530 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.323515    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:46:02.323571 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.323552    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:46:02.327437 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.327417    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:46:02.941867 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.941842    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 02:46:02.942714 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.942691    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 02:46:02.946262 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.946244    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 02:46:02.947143 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:02.947125    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 02:46:03.027872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:03.027842    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:46:03.030741 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:46:03.030723    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:47:05.154475 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.154438    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"]
May 26 02:47:05.155016 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.154894    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="registry" containerID="cri-o://1bb7210141ac85ff1357192ecea058a8056315379dd12bf0c9f85c844bfbf808" gracePeriod=30
May 26 02:47:05.155079 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.154982    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="ui" containerID="cri-o://9e766ba4bf561ca6944536da94828c9e8cbefc6701dfef75e77e9f790fa3d84d" gracePeriod=30
May 26 02:47:05.155079 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.155049    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="offline" containerID="cri-o://262747188eb8443138438cdd66b46b96aaf0dfb1e47a51b8ff9b78c2457c2385" gracePeriod=30
May 26 02:47:05.155183 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.155111    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="online" containerID="cri-o://f6dddf3f4bb855ddb22c4928d5a5c1bf4a0107d4e449b662f3d88615b7df8814" gracePeriod=30
May 26 02:47:05.420513 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.420424    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"]
May 26 02:47:05.420922 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.420883    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d947c490-d87e-4291-9891-1dcde8247601" containerName="console"
May 26 02:47:05.420922 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.420919    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d947c490-d87e-4291-9891-1dcde8247601" containerName="console"
May 26 02:47:05.421049 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.420996    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="d947c490-d87e-4291-9891-1dcde8247601" containerName="console"
May 26 02:47:05.424531 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.424510    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.426854 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.426831    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-8kp7t\""
May 26 02:47:05.435078 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.435056    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"]
May 26 02:47:05.466191 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.466160    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-q8w2x\" (UniqueName: \"kubernetes.io/projected/39150a74-7669-4741-82d0-db98d49f75bd-kube-api-access-q8w2x\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.466358 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.466220    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-ui-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.466358 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.466250    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/39150a74-7669-4741-82d0-db98d49f75bd-feast-data\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.466358 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.466305    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-registry-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.466358 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.466330    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-offline-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.466358 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.466349    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-online-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.567569 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.567528    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-q8w2x\" (UniqueName: \"kubernetes.io/projected/39150a74-7669-4741-82d0-db98d49f75bd-kube-api-access-q8w2x\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.567772 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.567604    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-ui-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.567772 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.567635    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/39150a74-7669-4741-82d0-db98d49f75bd-feast-data\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.567772 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.567706    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-registry-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.567772 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.567732    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-offline-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.567772 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.567759    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-online-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.568105 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:47:05.568082    2551 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 26 02:47:05.568170 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:47:05.568159    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-registry-tls podName:39150a74-7669-4741-82d0-db98d49f75bd nodeName:}" failed. No retries permitted until 2026-05-26 02:47:06.068134879 +0000 UTC m=+663.624923543 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-registry-tls") pod "feast-simple-feast-setup-8bf5d45fd-jkfg8" (UID: "39150a74-7669-4741-82d0-db98d49f75bd") : secret "feast-simple-feast-setup-registry-tls" not found
May 26 02:47:05.568395 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:47:05.568376    2551 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 26 02:47:05.568560 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:47:05.568549    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-ui-tls podName:39150a74-7669-4741-82d0-db98d49f75bd nodeName:}" failed. No retries permitted until 2026-05-26 02:47:06.068530367 +0000 UTC m=+663.625319028 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-ui-tls") pod "feast-simple-feast-setup-8bf5d45fd-jkfg8" (UID: "39150a74-7669-4741-82d0-db98d49f75bd") : secret "feast-simple-feast-setup-ui-tls" not found
May 26 02:47:05.568643 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.568572    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/39150a74-7669-4741-82d0-db98d49f75bd-feast-data\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.571087 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.571061    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-online-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.571217 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.571138    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-offline-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:05.576585 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:05.576558    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-q8w2x\" (UniqueName: \"kubernetes.io/projected/39150a74-7669-4741-82d0-db98d49f75bd-kube-api-access-q8w2x\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:06.072638 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:06.072600    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-registry-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:06.072801 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:06.072669    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-ui-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:06.075094 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:06.075064    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-registry-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:06.075208 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:06.075064    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-ui-tls\") pod \"feast-simple-feast-setup-8bf5d45fd-jkfg8\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") " pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:06.215469 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:06.215433    2551 generic.go:358] "Generic (PLEG): container finished" podID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerID="9e766ba4bf561ca6944536da94828c9e8cbefc6701dfef75e77e9f790fa3d84d" exitCode=0
May 26 02:47:06.215469 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:06.215459    2551 generic.go:358] "Generic (PLEG): container finished" podID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerID="f6dddf3f4bb855ddb22c4928d5a5c1bf4a0107d4e449b662f3d88615b7df8814" exitCode=0
May 26 02:47:06.215855 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:06.215501    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerDied","Data":"9e766ba4bf561ca6944536da94828c9e8cbefc6701dfef75e77e9f790fa3d84d"}
May 26 02:47:06.215855 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:06.215534    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerDied","Data":"f6dddf3f4bb855ddb22c4928d5a5c1bf4a0107d4e449b662f3d88615b7df8814"}
May 26 02:47:06.337323 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:06.337237    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:06.462497 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:06.462463    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"]
May 26 02:47:06.465291 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:47:06.465264    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod39150a74_7669_4741_82d0_db98d49f75bd.slice/crio-6939f1b000d7264f33978a227c27a50225ca5d7ecdc182b10eecaeba066d1595 WatchSource:0}: Error finding container 6939f1b000d7264f33978a227c27a50225ca5d7ecdc182b10eecaeba066d1595: Status 404 returned error can't find the container with id 6939f1b000d7264f33978a227c27a50225ca5d7ecdc182b10eecaeba066d1595
May 26 02:47:06.467057 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:06.467042    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 26 02:47:07.219968 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:07.219931    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerStarted","Data":"4fc0102ea46f6e99f28e049f9f97e545c6ef37421bffff95bdf6eceaf4612f12"}
May 26 02:47:07.219968 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:07.219972    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerStarted","Data":"6939f1b000d7264f33978a227c27a50225ca5d7ecdc182b10eecaeba066d1595"}
May 26 02:47:11.235269 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:11.235184    2551 generic.go:358] "Generic (PLEG): container finished" podID="39150a74-7669-4741-82d0-db98d49f75bd" containerID="4fc0102ea46f6e99f28e049f9f97e545c6ef37421bffff95bdf6eceaf4612f12" exitCode=0
May 26 02:47:11.235269 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:11.235245    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerDied","Data":"4fc0102ea46f6e99f28e049f9f97e545c6ef37421bffff95bdf6eceaf4612f12"}
May 26 02:47:12.240538 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:12.240504    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerStarted","Data":"98bfb29753f7a3e77830d6696a1664cf0ce62bdf48c69ea0f5554be2e28fa300"}
May 26 02:47:12.324058 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:12.324014    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 26 02:47:13.028831 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:13.028788    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 26 02:47:17.256453 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:17.256425    2551 generic.go:358] "Generic (PLEG): container finished" podID="39150a74-7669-4741-82d0-db98d49f75bd" containerID="98bfb29753f7a3e77830d6696a1664cf0ce62bdf48c69ea0f5554be2e28fa300" exitCode=0
May 26 02:47:17.256848 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:17.256480    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerDied","Data":"98bfb29753f7a3e77830d6696a1664cf0ce62bdf48c69ea0f5554be2e28fa300"}
May 26 02:47:18.263347 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.263308    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerStarted","Data":"bb75432200e236d8111fcccb49a99337bd5dc2a4fd329342b97ef8f5bd42b843"}
May 26 02:47:18.263347 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.263354    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerStarted","Data":"0ba975fe47fdc003f91c7cc2c20894373163f21c66f700cb2e8b0a66237be0fe"}
May 26 02:47:18.263883 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.263369    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerStarted","Data":"63ab781985aa52a9aa3478d26bef3f7ed4778b7bade7c767ff346d96d2506805"}
May 26 02:47:18.263883 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.263381    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerStarted","Data":"5a3adb4e67a9c45e5c6f82283eefb86149cbbdabb9e8a7cab31db6d68e711f9f"}
May 26 02:47:18.290640 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.290133    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podStartSLOduration=13.290112469 podStartE2EDuration="13.290112469s" podCreationTimestamp="2026-05-26 02:47:05 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:47:18.287114145 +0000 UTC m=+675.843902830" watchObservedRunningTime="2026-05-26 02:47:18.290112469 +0000 UTC m=+675.846901142"
May 26 02:47:18.338245 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.338206    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:18.338432 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.338256    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:18.338432 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.338273    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:18.338432 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.338284    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:18.340678 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.340625    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused"
May 26 02:47:18.340678 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.340655    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 26 02:47:18.340867 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.340702    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 26 02:47:18.340867 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:18.340738    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.29:8816: connect: connection refused"
May 26 02:47:21.338479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:21.338335    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.29:8816: connect: connection refused"
May 26 02:47:21.338479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:21.338387    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 26 02:47:21.338479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:21.338385    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 26 02:47:21.338479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:21.338388    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.29:6571: connect: connection refused"
May 26 02:47:22.323720 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:22.323676    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 26 02:47:23.028426 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:23.028390    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 26 02:47:24.339194 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:24.339153    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:24.339561 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:24.339353    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:24.339561 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:24.339465    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:24.339834 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:24.339810    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:24.340001 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:24.339844    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:24.340001 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:24.339860    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:24.340084 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:24.340025    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:24.340179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:24.340161    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:24.340318 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:24.340300    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:24.343798 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:24.343778    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:25.293427 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:25.293395    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:25.296434 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:25.296414    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:47:32.324051 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:32.323951    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.28:8443: connect: connection refused"
May 26 02:47:32.324473 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:32.324300    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:47:33.028378 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:33.028336    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 26 02:47:33.028553 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:33.028455    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:47:35.325290 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.325264    2551 generic.go:358] "Generic (PLEG): container finished" podID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerID="262747188eb8443138438cdd66b46b96aaf0dfb1e47a51b8ff9b78c2457c2385" exitCode=137
May 26 02:47:35.325290 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.325288    2551 generic.go:358] "Generic (PLEG): container finished" podID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerID="1bb7210141ac85ff1357192ecea058a8056315379dd12bf0c9f85c844bfbf808" exitCode=137
May 26 02:47:35.325665 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.325333    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerDied","Data":"262747188eb8443138438cdd66b46b96aaf0dfb1e47a51b8ff9b78c2457c2385"}
May 26 02:47:35.325665 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.325365    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerDied","Data":"1bb7210141ac85ff1357192ecea058a8056315379dd12bf0c9f85c844bfbf808"}
May 26 02:47:35.807235 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.807207    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:47:35.957384 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.957302    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-offline-tls\") pod \"99af60ce-315a-481c-a29e-9c8e7e9de804\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") "
May 26 02:47:35.957567 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.957390    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jhpq8\" (UniqueName: \"kubernetes.io/projected/99af60ce-315a-481c-a29e-9c8e7e9de804-kube-api-access-jhpq8\") pod \"99af60ce-315a-481c-a29e-9c8e7e9de804\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") "
May 26 02:47:35.957567 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.957408    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-ui-tls\") pod \"99af60ce-315a-481c-a29e-9c8e7e9de804\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") "
May 26 02:47:35.957567 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.957430    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-online-tls\") pod \"99af60ce-315a-481c-a29e-9c8e7e9de804\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") "
May 26 02:47:35.957567 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.957447    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/99af60ce-315a-481c-a29e-9c8e7e9de804-feast-data\") pod \"99af60ce-315a-481c-a29e-9c8e7e9de804\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") "
May 26 02:47:35.957567 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.957516    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-registry-tls\") pod \"99af60ce-315a-481c-a29e-9c8e7e9de804\" (UID: \"99af60ce-315a-481c-a29e-9c8e7e9de804\") "
May 26 02:47:35.958024 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.957996    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/99af60ce-315a-481c-a29e-9c8e7e9de804-feast-data" (OuterVolumeSpecName: "feast-data") pod "99af60ce-315a-481c-a29e-9c8e7e9de804" (UID: "99af60ce-315a-481c-a29e-9c8e7e9de804"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 02:47:35.959715 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.959685    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/99af60ce-315a-481c-a29e-9c8e7e9de804-kube-api-access-jhpq8" (OuterVolumeSpecName: "kube-api-access-jhpq8") pod "99af60ce-315a-481c-a29e-9c8e7e9de804" (UID: "99af60ce-315a-481c-a29e-9c8e7e9de804"). InnerVolumeSpecName "kube-api-access-jhpq8". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:47:35.959827 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.959708    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "99af60ce-315a-481c-a29e-9c8e7e9de804" (UID: "99af60ce-315a-481c-a29e-9c8e7e9de804"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:47:35.959827 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.959732    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "99af60ce-315a-481c-a29e-9c8e7e9de804" (UID: "99af60ce-315a-481c-a29e-9c8e7e9de804"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:47:35.960015 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.959936    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-online-tls" (OuterVolumeSpecName: "online-tls") pod "99af60ce-315a-481c-a29e-9c8e7e9de804" (UID: "99af60ce-315a-481c-a29e-9c8e7e9de804"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:47:35.960015 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:35.959953    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "99af60ce-315a-481c-a29e-9c8e7e9de804" (UID: "99af60ce-315a-481c-a29e-9c8e7e9de804"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:47:36.058220 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.058189    2551 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-offline-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:47:36.058220 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.058215    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jhpq8\" (UniqueName: \"kubernetes.io/projected/99af60ce-315a-481c-a29e-9c8e7e9de804-kube-api-access-jhpq8\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:47:36.058220 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.058225    2551 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-ui-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:47:36.058437 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.058236    2551 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-online-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:47:36.058437 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.058246    2551 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/99af60ce-315a-481c-a29e-9c8e7e9de804-feast-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:47:36.058437 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.058253    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/99af60ce-315a-481c-a29e-9c8e7e9de804-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:47:36.330516 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.330482    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9" event={"ID":"99af60ce-315a-481c-a29e-9c8e7e9de804","Type":"ContainerDied","Data":"b088f1c34b3b2a3d550e84b4b27bc138534d34c942b54f87a2abd2dc7b4fdcb4"}
May 26 02:47:36.330888 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.330531    2551 scope.go:117] "RemoveContainer" containerID="9e766ba4bf561ca6944536da94828c9e8cbefc6701dfef75e77e9f790fa3d84d"
May 26 02:47:36.330888 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.330558    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"
May 26 02:47:36.339087 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.339067    2551 scope.go:117] "RemoveContainer" containerID="262747188eb8443138438cdd66b46b96aaf0dfb1e47a51b8ff9b78c2457c2385"
May 26 02:47:36.346387 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.346370    2551 scope.go:117] "RemoveContainer" containerID="f6dddf3f4bb855ddb22c4928d5a5c1bf4a0107d4e449b662f3d88615b7df8814"
May 26 02:47:36.353570 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.353536    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"]
May 26 02:47:36.355120 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.355097    2551 scope.go:117] "RemoveContainer" containerID="1bb7210141ac85ff1357192ecea058a8056315379dd12bf0c9f85c844bfbf808"
May 26 02:47:36.362063 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.360874    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8bf5d45fd-555g9"]
May 26 02:47:36.368801 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.368776    2551 scope.go:117] "RemoveContainer" containerID="42551d10439f1d39b9cd4283318c5ad0a688fcafaf30cabf03ff61e15a7ae315"
May 26 02:47:36.375715 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:36.375697    2551 scope.go:117] "RemoveContainer" containerID="eb1415e81fb8ae975162f96baa85b7a83aed13558d544b6cccf67da7d765488f"
May 26 02:47:37.044817 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:47:37.044784    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" path="/var/lib/kubelet/pods/99af60ce-315a-481c-a29e-9c8e7e9de804/volumes"
May 26 02:48:27.113966 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.113926    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"]
May 26 02:48:27.114479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114378    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="offline"
May 26 02:48:27.114479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114394    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="offline"
May 26 02:48:27.114479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114410    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="feast-apply"
May 26 02:48:27.114479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114419    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="feast-apply"
May 26 02:48:27.114479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114431    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="online"
May 26 02:48:27.114479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114439    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="online"
May 26 02:48:27.114479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114465    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="feast-init"
May 26 02:48:27.114479 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114473    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="feast-init"
May 26 02:48:27.114872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114486    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="registry"
May 26 02:48:27.114872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114495    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="registry"
May 26 02:48:27.114872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114511    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="ui"
May 26 02:48:27.114872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114519    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="ui"
May 26 02:48:27.114872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114585    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="online"
May 26 02:48:27.114872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114595    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="registry"
May 26 02:48:27.114872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114605    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="ui"
May 26 02:48:27.114872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.114619    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="99af60ce-315a-481c-a29e-9c8e7e9de804" containerName="offline"
May 26 02:48:27.118115 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.118098    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.120520 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.120498    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-6nd5v\""
May 26 02:48:27.120649 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.120522    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
May 26 02:48:27.120649 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.120543    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 26 02:48:27.120649 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.120636    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 26 02:48:27.120928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.120888    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 26 02:48:27.121025 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.120940    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 26 02:48:27.121511 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.121494    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 26 02:48:27.121610 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.121510    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 26 02:48:27.128955 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.128930    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"]
May 26 02:48:27.197247 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.197212    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-registry-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.197441 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.197260    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-ui-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.197441 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.197294    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-online-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.197441 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.197363    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pwqsp\" (UniqueName: \"kubernetes.io/projected/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-kube-api-access-pwqsp\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.197441 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.197388    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-offline-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.197441 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.197406    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-feast-data\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.298420 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.298386    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pwqsp\" (UniqueName: \"kubernetes.io/projected/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-kube-api-access-pwqsp\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.298605 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.298434    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-offline-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.298605 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.298463    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-feast-data\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.298605 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.298519    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-registry-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.298605 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.298556    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-ui-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.298605 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.298589    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-online-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.298858 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:48:27.298692    2551 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-online-tls: secret "feast-simple-feast-remote-setup-online-tls" not found
May 26 02:48:27.298858 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:48:27.298757    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-online-tls podName:43a3b7b8-93df-4fa6-ad87-2dba99df5d8a nodeName:}" failed. No retries permitted until 2026-05-26 02:48:27.798737527 +0000 UTC m=+745.355526174 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-online-tls") pod "feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" (UID: "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a") : secret "feast-simple-feast-remote-setup-online-tls" not found
May 26 02:48:27.299003 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.298960    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-feast-data\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.299283 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.299260    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-registry-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.301078 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.301060    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-ui-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.301196 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.301178    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-offline-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.308706 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.308681    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pwqsp\" (UniqueName: \"kubernetes.io/projected/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-kube-api-access-pwqsp\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.801990 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.801939    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-online-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:27.804498 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:27.804463    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-online-tls\") pod \"feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:28.029657 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:28.029622    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:28.159039 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:28.159004    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"]
May 26 02:48:28.162610 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:48:28.162584    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod43a3b7b8_93df_4fa6_ad87_2dba99df5d8a.slice/crio-58d09fb7176ae76e82d0dd8770ae87298b738951e9f3a3c6c792e2a4656b724a WatchSource:0}: Error finding container 58d09fb7176ae76e82d0dd8770ae87298b738951e9f3a3c6c792e2a4656b724a: Status 404 returned error can't find the container with id 58d09fb7176ae76e82d0dd8770ae87298b738951e9f3a3c6c792e2a4656b724a
May 26 02:48:28.483796 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:28.483707    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" event={"ID":"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a","Type":"ContainerStarted","Data":"8f41702e30bf75ca02c022799fe2dd141074f5c14ad2c6ab143a485ee2f1b388"}
May 26 02:48:28.483796 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:28.483752    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" event={"ID":"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a","Type":"ContainerStarted","Data":"58d09fb7176ae76e82d0dd8770ae87298b738951e9f3a3c6c792e2a4656b724a"}
May 26 02:48:33.499690 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:33.499657    2551 generic.go:358] "Generic (PLEG): container finished" podID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerID="8f41702e30bf75ca02c022799fe2dd141074f5c14ad2c6ab143a485ee2f1b388" exitCode=0
May 26 02:48:33.500126 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:33.499732    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" event={"ID":"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a","Type":"ContainerDied","Data":"8f41702e30bf75ca02c022799fe2dd141074f5c14ad2c6ab143a485ee2f1b388"}
May 26 02:48:34.504374 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:34.504343    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" event={"ID":"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a","Type":"ContainerStarted","Data":"8477dae70269d13c56018243280dad2f0b4f3b03328f483905cfee4594cedb46"}
May 26 02:48:39.523097 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:39.523061    2551 generic.go:358] "Generic (PLEG): container finished" podID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerID="8477dae70269d13c56018243280dad2f0b4f3b03328f483905cfee4594cedb46" exitCode=0
May 26 02:48:39.523568 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:39.523139    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" event={"ID":"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a","Type":"ContainerDied","Data":"8477dae70269d13c56018243280dad2f0b4f3b03328f483905cfee4594cedb46"}
May 26 02:48:40.529813 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:40.529774    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" event={"ID":"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a","Type":"ContainerStarted","Data":"01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649"}
May 26 02:48:40.530229 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:40.529827    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" event={"ID":"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a","Type":"ContainerStarted","Data":"8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be"}
May 26 02:48:40.530229 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:40.529843    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" event={"ID":"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a","Type":"ContainerStarted","Data":"d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e"}
May 26 02:48:40.553572 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:40.553516    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podStartSLOduration=13.553499634 podStartE2EDuration="13.553499634s" podCreationTimestamp="2026-05-26 02:48:27 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:48:40.551207543 +0000 UTC m=+758.107996211" watchObservedRunningTime="2026-05-26 02:48:40.553499634 +0000 UTC m=+758.110288302"
May 26 02:48:43.029882 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:43.029842    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:43.029882 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:43.029890    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:43.030441 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:43.029925    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:43.031372 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:43.031340    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.30:8816: connect: connection refused"
May 26 02:48:43.031507 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:43.031386    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 26 02:48:43.031507 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:43.031390    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 26 02:48:46.031204 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:46.031175    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:46.031584 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:46.031286    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:46.031754 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:46.031734    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:46.031807 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:46.031762    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:46.032056 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:46.032030    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:46.032154 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:46.032082    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:46.035446 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:46.035427    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:46.551110 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:46.551082    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:48:46.553957 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:48:46.553937    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:49:48.357062 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:48.357030    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"]
May 26 02:49:48.357584 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:48.357345    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="online" containerID="cri-o://d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e" gracePeriod=30
May 26 02:49:48.357584 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:48.357378    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="ui" containerID="cri-o://01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649" gracePeriod=30
May 26 02:49:48.357584 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:48.357399    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="offline" containerID="cri-o://8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be" gracePeriod=30
May 26 02:49:48.475390 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:48.475356    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"]
May 26 02:49:48.475778 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:48.475749    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="registry" containerID="cri-o://5a3adb4e67a9c45e5c6f82283eefb86149cbbdabb9e8a7cab31db6d68e711f9f" gracePeriod=30
May 26 02:49:48.476030 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:48.475792    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="ui" containerID="cri-o://bb75432200e236d8111fcccb49a99337bd5dc2a4fd329342b97ef8f5bd42b843" gracePeriod=30
May 26 02:49:48.476161 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:48.475822    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="offline" containerID="cri-o://0ba975fe47fdc003f91c7cc2c20894373163f21c66f700cb2e8b0a66237be0fe" gracePeriod=30
May 26 02:49:48.476161 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:48.475850    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="online" containerID="cri-o://63ab781985aa52a9aa3478d26bef3f7ed4778b7bade7c767ff346d96d2506805" gracePeriod=30
May 26 02:49:49.748694 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:49.748659    2551 generic.go:358] "Generic (PLEG): container finished" podID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerID="01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649" exitCode=0
May 26 02:49:49.748694 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:49.748685    2551 generic.go:358] "Generic (PLEG): container finished" podID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerID="d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e" exitCode=0
May 26 02:49:49.749116 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:49.748728    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" event={"ID":"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a","Type":"ContainerDied","Data":"01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649"}
May 26 02:49:49.749116 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:49.748762    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" event={"ID":"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a","Type":"ContainerDied","Data":"d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e"}
May 26 02:49:49.751015 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:49.750995    2551 generic.go:358] "Generic (PLEG): container finished" podID="39150a74-7669-4741-82d0-db98d49f75bd" containerID="bb75432200e236d8111fcccb49a99337bd5dc2a4fd329342b97ef8f5bd42b843" exitCode=0
May 26 02:49:49.751015 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:49.751015    2551 generic.go:358] "Generic (PLEG): container finished" podID="39150a74-7669-4741-82d0-db98d49f75bd" containerID="63ab781985aa52a9aa3478d26bef3f7ed4778b7bade7c767ff346d96d2506805" exitCode=0
May 26 02:49:49.751163 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:49.751065    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerDied","Data":"bb75432200e236d8111fcccb49a99337bd5dc2a4fd329342b97ef8f5bd42b843"}
May 26 02:49:49.751163 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:49.751092    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerDied","Data":"63ab781985aa52a9aa3478d26bef3f7ed4778b7bade7c767ff346d96d2506805"}
May 26 02:49:54.340954 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:54.340877    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 26 02:49:55.294461 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:55.294414    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 26 02:49:56.032783 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:56.032737    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 26 02:49:56.551570 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:49:56.551525    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 26 02:50:04.340666 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:04.340603    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 26 02:50:05.294266 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:05.294227    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 26 02:50:06.031995 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:06.031951    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 26 02:50:06.551586 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:06.551545    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 26 02:50:14.340657 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:14.340604    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.29:8443: connect: connection refused"
May 26 02:50:14.341146 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:14.340750    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:50:15.293789 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:15.293741    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="online" probeResult="failure" output="Get \"https://10.134.0.29:6567/health\": dial tcp 10.134.0.29:6567: connect: connection refused"
May 26 02:50:15.294012 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:15.293883    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:50:16.032004 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:16.031959    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.30:8443: connect: connection refused"
May 26 02:50:16.032486 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:16.032083    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:50:16.551873 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:16.551832    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.30:6567/health\": dial tcp 10.134.0.30:6567: connect: connection refused"
May 26 02:50:16.552058 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:16.551976    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:50:18.597732 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.597702    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:50:18.636755 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.636671    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-online-tls\") pod \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") "
May 26 02:50:18.636755 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.636743    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-offline-tls\") pod \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") "
May 26 02:50:18.637023 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.636774    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-ui-tls\") pod \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") "
May 26 02:50:18.637023 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.636862    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-feast-data\") pod \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") "
May 26 02:50:18.637023 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.636957    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pwqsp\" (UniqueName: \"kubernetes.io/projected/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-kube-api-access-pwqsp\") pod \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") "
May 26 02:50:18.637023 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.636997    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-registry-tls\") pod \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\" (UID: \"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a\") "
May 26 02:50:18.637572 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.637520    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" (UID: "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 02:50:18.637703 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.637586    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-feast-data" (OuterVolumeSpecName: "feast-data") pod "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" (UID: "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 02:50:18.639206 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.639175    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-kube-api-access-pwqsp" (OuterVolumeSpecName: "kube-api-access-pwqsp") pod "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" (UID: "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a"). InnerVolumeSpecName "kube-api-access-pwqsp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:50:18.639351 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.639325    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" (UID: "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:50:18.639442 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.639357    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" (UID: "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:50:18.639442 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.639384    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-online-tls" (OuterVolumeSpecName: "online-tls") pod "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" (UID: "43a3b7b8-93df-4fa6-ad87-2dba99df5d8a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:50:18.738270 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.738220    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pwqsp\" (UniqueName: \"kubernetes.io/projected/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-kube-api-access-pwqsp\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:50:18.738270 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.738265    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:50:18.738270 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.738276    2551 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-online-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:50:18.738270 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.738287    2551 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-offline-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:50:18.738541 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.738296    2551 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-ui-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:50:18.738541 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.738304    2551 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a-feast-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:50:18.838117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.838086    2551 generic.go:358] "Generic (PLEG): container finished" podID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerID="8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be" exitCode=137
May 26 02:50:18.838288 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.838153    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"
May 26 02:50:18.838288 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.838157    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" event={"ID":"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a","Type":"ContainerDied","Data":"8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be"}
May 26 02:50:18.838288 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.838186    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk" event={"ID":"43a3b7b8-93df-4fa6-ad87-2dba99df5d8a","Type":"ContainerDied","Data":"58d09fb7176ae76e82d0dd8770ae87298b738951e9f3a3c6c792e2a4656b724a"}
May 26 02:50:18.838288 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.838203    2551 scope.go:117] "RemoveContainer" containerID="01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649"
May 26 02:50:18.841041 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.841018    2551 generic.go:358] "Generic (PLEG): container finished" podID="39150a74-7669-4741-82d0-db98d49f75bd" containerID="0ba975fe47fdc003f91c7cc2c20894373163f21c66f700cb2e8b0a66237be0fe" exitCode=137
May 26 02:50:18.841041 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.841038    2551 generic.go:358] "Generic (PLEG): container finished" podID="39150a74-7669-4741-82d0-db98d49f75bd" containerID="5a3adb4e67a9c45e5c6f82283eefb86149cbbdabb9e8a7cab31db6d68e711f9f" exitCode=137
May 26 02:50:18.841179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.841083    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerDied","Data":"0ba975fe47fdc003f91c7cc2c20894373163f21c66f700cb2e8b0a66237be0fe"}
May 26 02:50:18.841179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.841106    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerDied","Data":"5a3adb4e67a9c45e5c6f82283eefb86149cbbdabb9e8a7cab31db6d68e711f9f"}
May 26 02:50:18.847226 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.847208    2551 scope.go:117] "RemoveContainer" containerID="8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be"
May 26 02:50:18.854571 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.854552    2551 scope.go:117] "RemoveContainer" containerID="d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e"
May 26 02:50:18.860782 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.860757    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"]
May 26 02:50:18.864069 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.864053    2551 scope.go:117] "RemoveContainer" containerID="8477dae70269d13c56018243280dad2f0b4f3b03328f483905cfee4594cedb46"
May 26 02:50:18.865157 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.865140    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5f5f5f84bc-qjhpk"]
May 26 02:50:18.870954 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.870933    2551 scope.go:117] "RemoveContainer" containerID="8f41702e30bf75ca02c022799fe2dd141074f5c14ad2c6ab143a485ee2f1b388"
May 26 02:50:18.877699 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.877682    2551 scope.go:117] "RemoveContainer" containerID="01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649"
May 26 02:50:18.877956 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:50:18.877938    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649\": container with ID starting with 01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649 not found: ID does not exist" containerID="01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649"
May 26 02:50:18.878009 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.877965    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649"} err="failed to get container status \"01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649\": rpc error: code = NotFound desc = could not find container \"01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649\": container with ID starting with 01c8ecf0ab36f21b0289d88923d476c4b7626c2fada74c21a13698cfe9d4c649 not found: ID does not exist"
May 26 02:50:18.878009 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.877983    2551 scope.go:117] "RemoveContainer" containerID="8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be"
May 26 02:50:18.878200 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:50:18.878185    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be\": container with ID starting with 8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be not found: ID does not exist" containerID="8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be"
May 26 02:50:18.878241 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.878204    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be"} err="failed to get container status \"8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be\": rpc error: code = NotFound desc = could not find container \"8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be\": container with ID starting with 8685e59ec3b77a1ad810dc4e20f58a4e913244ee419d0ee91b13eb38371024be not found: ID does not exist"
May 26 02:50:18.878241 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.878218    2551 scope.go:117] "RemoveContainer" containerID="d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e"
May 26 02:50:18.878415 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:50:18.878399    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e\": container with ID starting with d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e not found: ID does not exist" containerID="d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e"
May 26 02:50:18.878462 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.878419    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e"} err="failed to get container status \"d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e\": rpc error: code = NotFound desc = could not find container \"d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e\": container with ID starting with d6e9e7178bd5601fa430538d3f751cbbba48a7f33204e8d21ed4d385ff951c0e not found: ID does not exist"
May 26 02:50:18.878462 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.878432    2551 scope.go:117] "RemoveContainer" containerID="8477dae70269d13c56018243280dad2f0b4f3b03328f483905cfee4594cedb46"
May 26 02:50:18.878613 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:50:18.878599    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8477dae70269d13c56018243280dad2f0b4f3b03328f483905cfee4594cedb46\": container with ID starting with 8477dae70269d13c56018243280dad2f0b4f3b03328f483905cfee4594cedb46 not found: ID does not exist" containerID="8477dae70269d13c56018243280dad2f0b4f3b03328f483905cfee4594cedb46"
May 26 02:50:18.878650 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.878618    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8477dae70269d13c56018243280dad2f0b4f3b03328f483905cfee4594cedb46"} err="failed to get container status \"8477dae70269d13c56018243280dad2f0b4f3b03328f483905cfee4594cedb46\": rpc error: code = NotFound desc = could not find container \"8477dae70269d13c56018243280dad2f0b4f3b03328f483905cfee4594cedb46\": container with ID starting with 8477dae70269d13c56018243280dad2f0b4f3b03328f483905cfee4594cedb46 not found: ID does not exist"
May 26 02:50:18.878650 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.878629    2551 scope.go:117] "RemoveContainer" containerID="8f41702e30bf75ca02c022799fe2dd141074f5c14ad2c6ab143a485ee2f1b388"
May 26 02:50:18.878828 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:50:18.878813    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8f41702e30bf75ca02c022799fe2dd141074f5c14ad2c6ab143a485ee2f1b388\": container with ID starting with 8f41702e30bf75ca02c022799fe2dd141074f5c14ad2c6ab143a485ee2f1b388 not found: ID does not exist" containerID="8f41702e30bf75ca02c022799fe2dd141074f5c14ad2c6ab143a485ee2f1b388"
May 26 02:50:18.878878 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:18.878831    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8f41702e30bf75ca02c022799fe2dd141074f5c14ad2c6ab143a485ee2f1b388"} err="failed to get container status \"8f41702e30bf75ca02c022799fe2dd141074f5c14ad2c6ab143a485ee2f1b388\": rpc error: code = NotFound desc = could not find container \"8f41702e30bf75ca02c022799fe2dd141074f5c14ad2c6ab143a485ee2f1b388\": container with ID starting with 8f41702e30bf75ca02c022799fe2dd141074f5c14ad2c6ab143a485ee2f1b388 not found: ID does not exist"
May 26 02:50:19.044942 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.044893    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" path="/var/lib/kubelet/pods/43a3b7b8-93df-4fa6-ad87-2dba99df5d8a/volumes"
May 26 02:50:19.126514 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.126494    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:50:19.140710 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.140685    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/39150a74-7669-4741-82d0-db98d49f75bd-feast-data\") pod \"39150a74-7669-4741-82d0-db98d49f75bd\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") "
May 26 02:50:19.140828 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.140715    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-online-tls\") pod \"39150a74-7669-4741-82d0-db98d49f75bd\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") "
May 26 02:50:19.140828 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.140744    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-ui-tls\") pod \"39150a74-7669-4741-82d0-db98d49f75bd\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") "
May 26 02:50:19.140828 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.140794    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-q8w2x\" (UniqueName: \"kubernetes.io/projected/39150a74-7669-4741-82d0-db98d49f75bd-kube-api-access-q8w2x\") pod \"39150a74-7669-4741-82d0-db98d49f75bd\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") "
May 26 02:50:19.141023 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.140838    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-offline-tls\") pod \"39150a74-7669-4741-82d0-db98d49f75bd\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") "
May 26 02:50:19.141023 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.140966    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-registry-tls\") pod \"39150a74-7669-4741-82d0-db98d49f75bd\" (UID: \"39150a74-7669-4741-82d0-db98d49f75bd\") "
May 26 02:50:19.141285 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.141252    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/39150a74-7669-4741-82d0-db98d49f75bd-feast-data" (OuterVolumeSpecName: "feast-data") pod "39150a74-7669-4741-82d0-db98d49f75bd" (UID: "39150a74-7669-4741-82d0-db98d49f75bd"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 02:50:19.142999 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.142926    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "39150a74-7669-4741-82d0-db98d49f75bd" (UID: "39150a74-7669-4741-82d0-db98d49f75bd"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:50:19.143089 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.143023    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "39150a74-7669-4741-82d0-db98d49f75bd" (UID: "39150a74-7669-4741-82d0-db98d49f75bd"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:50:19.143089 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.143037    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "39150a74-7669-4741-82d0-db98d49f75bd" (UID: "39150a74-7669-4741-82d0-db98d49f75bd"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:50:19.143269 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.143255    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/39150a74-7669-4741-82d0-db98d49f75bd-kube-api-access-q8w2x" (OuterVolumeSpecName: "kube-api-access-q8w2x") pod "39150a74-7669-4741-82d0-db98d49f75bd" (UID: "39150a74-7669-4741-82d0-db98d49f75bd"). InnerVolumeSpecName "kube-api-access-q8w2x". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:50:19.143328 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.143308    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-online-tls" (OuterVolumeSpecName: "online-tls") pod "39150a74-7669-4741-82d0-db98d49f75bd" (UID: "39150a74-7669-4741-82d0-db98d49f75bd"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:50:19.242381 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.242349    2551 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-offline-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:50:19.242381 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.242377    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:50:19.242381 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.242390    2551 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/39150a74-7669-4741-82d0-db98d49f75bd-feast-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:50:19.242605 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.242402    2551 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-online-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:50:19.242605 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.242412    2551 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/39150a74-7669-4741-82d0-db98d49f75bd-ui-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:50:19.242605 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.242425    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-q8w2x\" (UniqueName: \"kubernetes.io/projected/39150a74-7669-4741-82d0-db98d49f75bd-kube-api-access-q8w2x\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:50:19.846742 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.846713    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8" event={"ID":"39150a74-7669-4741-82d0-db98d49f75bd","Type":"ContainerDied","Data":"6939f1b000d7264f33978a227c27a50225ca5d7ecdc182b10eecaeba066d1595"}
May 26 02:50:19.846742 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.846754    2551 scope.go:117] "RemoveContainer" containerID="bb75432200e236d8111fcccb49a99337bd5dc2a4fd329342b97ef8f5bd42b843"
May 26 02:50:19.847226 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.846761    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"
May 26 02:50:19.855068 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.855047    2551 scope.go:117] "RemoveContainer" containerID="0ba975fe47fdc003f91c7cc2c20894373163f21c66f700cb2e8b0a66237be0fe"
May 26 02:50:19.861954 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.861919    2551 scope.go:117] "RemoveContainer" containerID="63ab781985aa52a9aa3478d26bef3f7ed4778b7bade7c767ff346d96d2506805"
May 26 02:50:19.871062 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.871043    2551 scope.go:117] "RemoveContainer" containerID="5a3adb4e67a9c45e5c6f82283eefb86149cbbdabb9e8a7cab31db6d68e711f9f"
May 26 02:50:19.874570 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.874546    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"]
May 26 02:50:19.876001 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.875980    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-8bf5d45fd-jkfg8"]
May 26 02:50:19.879704 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.879687    2551 scope.go:117] "RemoveContainer" containerID="98bfb29753f7a3e77830d6696a1664cf0ce62bdf48c69ea0f5554be2e28fa300"
May 26 02:50:19.886431 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:19.886414    2551 scope.go:117] "RemoveContainer" containerID="4fc0102ea46f6e99f28e049f9f97e545c6ef37421bffff95bdf6eceaf4612f12"
May 26 02:50:21.044328 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:21.044294    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="39150a74-7669-4741-82d0-db98d49f75bd" path="/var/lib/kubelet/pods/39150a74-7669-4741-82d0-db98d49f75bd/volumes"
May 26 02:50:31.172383 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172350    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-ms7s5"]
May 26 02:50:31.172820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172771    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="feast-apply"
May 26 02:50:31.172820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172787    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="feast-apply"
May 26 02:50:31.172820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172797    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="online"
May 26 02:50:31.172820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172805    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="online"
May 26 02:50:31.172820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172816    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="online"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172825    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="online"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172837    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="offline"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172845    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="offline"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172855    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="feast-init"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172862    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="feast-init"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172882    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="ui"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172890    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="ui"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172930    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="offline"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172939    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="offline"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172948    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="ui"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172956    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="ui"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172966    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="feast-apply"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172974    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="feast-apply"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172984    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="feast-init"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.172992    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="feast-init"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.173000    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="registry"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.173008    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="registry"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.173104    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="offline"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.173118    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="ui"
May 26 02:50:31.173117 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.173128    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="online"
May 26 02:50:31.174084 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.173139    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="ui"
May 26 02:50:31.174084 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.173149    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="43a3b7b8-93df-4fa6-ad87-2dba99df5d8a" containerName="offline"
May 26 02:50:31.174084 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.173161    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="registry"
May 26 02:50:31.174084 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.173173    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="39150a74-7669-4741-82d0-db98d49f75bd" containerName="online"
May 26 02:50:31.177962 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.177943    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-ms7s5"
May 26 02:50:31.180358 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.180333    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-k27h7\""
May 26 02:50:31.180469 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.180433    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
May 26 02:50:31.180469 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.180459    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 26 02:50:31.181176 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.181154    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 26 02:50:31.181406 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.181384    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-ms7s5"]
May 26 02:50:31.240462 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.240432    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/86bf3be7-b7e7-4a26-aa51-f6591025497a-postgresdata\") pod \"postgres-7c7656cc6d-ms7s5\" (UID: \"86bf3be7-b7e7-4a26-aa51-f6591025497a\") " pod="test-ns-feast/postgres-7c7656cc6d-ms7s5"
May 26 02:50:31.240608 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.240475    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fgdqk\" (UniqueName: \"kubernetes.io/projected/86bf3be7-b7e7-4a26-aa51-f6591025497a-kube-api-access-fgdqk\") pod \"postgres-7c7656cc6d-ms7s5\" (UID: \"86bf3be7-b7e7-4a26-aa51-f6591025497a\") " pod="test-ns-feast/postgres-7c7656cc6d-ms7s5"
May 26 02:50:31.340851 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.340822    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/86bf3be7-b7e7-4a26-aa51-f6591025497a-postgresdata\") pod \"postgres-7c7656cc6d-ms7s5\" (UID: \"86bf3be7-b7e7-4a26-aa51-f6591025497a\") " pod="test-ns-feast/postgres-7c7656cc6d-ms7s5"
May 26 02:50:31.341015 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.340876    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fgdqk\" (UniqueName: \"kubernetes.io/projected/86bf3be7-b7e7-4a26-aa51-f6591025497a-kube-api-access-fgdqk\") pod \"postgres-7c7656cc6d-ms7s5\" (UID: \"86bf3be7-b7e7-4a26-aa51-f6591025497a\") " pod="test-ns-feast/postgres-7c7656cc6d-ms7s5"
May 26 02:50:31.341280 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.341262    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/86bf3be7-b7e7-4a26-aa51-f6591025497a-postgresdata\") pod \"postgres-7c7656cc6d-ms7s5\" (UID: \"86bf3be7-b7e7-4a26-aa51-f6591025497a\") " pod="test-ns-feast/postgres-7c7656cc6d-ms7s5"
May 26 02:50:31.349221 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.349196    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fgdqk\" (UniqueName: \"kubernetes.io/projected/86bf3be7-b7e7-4a26-aa51-f6591025497a-kube-api-access-fgdqk\") pod \"postgres-7c7656cc6d-ms7s5\" (UID: \"86bf3be7-b7e7-4a26-aa51-f6591025497a\") " pod="test-ns-feast/postgres-7c7656cc6d-ms7s5"
May 26 02:50:31.497636 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.497554    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-ms7s5"
May 26 02:50:31.620333 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.620302    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-ms7s5"]
May 26 02:50:31.622730 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:50:31.622704    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod86bf3be7_b7e7_4a26_aa51_f6591025497a.slice/crio-835548df1cdca918c13a8ae511d49bd63ab757a0a50b45aeb44b1a1c2e449e44 WatchSource:0}: Error finding container 835548df1cdca918c13a8ae511d49bd63ab757a0a50b45aeb44b1a1c2e449e44: Status 404 returned error can't find the container with id 835548df1cdca918c13a8ae511d49bd63ab757a0a50b45aeb44b1a1c2e449e44
May 26 02:50:31.883240 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:31.883205    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-ms7s5" event={"ID":"86bf3be7-b7e7-4a26-aa51-f6591025497a","Type":"ContainerStarted","Data":"835548df1cdca918c13a8ae511d49bd63ab757a0a50b45aeb44b1a1c2e449e44"}
May 26 02:50:37.903812 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:37.903727    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-ms7s5" event={"ID":"86bf3be7-b7e7-4a26-aa51-f6591025497a","Type":"ContainerStarted","Data":"d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2"}
May 26 02:50:37.918294 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:37.918243    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/postgres-7c7656cc6d-ms7s5" podStartSLOduration=0.903776535 podStartE2EDuration="6.918224997s" podCreationTimestamp="2026-05-26 02:50:31 +0000 UTC" firstStartedPulling="2026-05-26 02:50:31.624476696 +0000 UTC m=+869.181265343" lastFinishedPulling="2026-05-26 02:50:37.638925158 +0000 UTC m=+875.195713805" observedRunningTime="2026-05-26 02:50:37.917160159 +0000 UTC m=+875.473948829" watchObservedRunningTime="2026-05-26 02:50:37.918224997 +0000 UTC m=+875.475013668"
May 26 02:50:41.874024 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.873987    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"]
May 26 02:50:41.881364 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.881339    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:41.883841 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.883818    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\""
May 26 02:50:41.883999 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.883893    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 26 02:50:41.883999 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.883938    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\""
May 26 02:50:41.884257 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.884235    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\""
May 26 02:50:41.884366 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.884276    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-79mv5\""
May 26 02:50:41.889955 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.889929    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"]
May 26 02:50:41.933839 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.933811    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6d6c\" (UniqueName: \"kubernetes.io/projected/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-kube-api-access-c6d6c\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:41.933990 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.933868    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-offline-tls\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:41.934048 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.933972    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-online-tls\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:41.934108 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.934094    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-feast-data\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:41.934162 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:41.934128    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-registry-tls\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.035526 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.035498    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-offline-tls\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.035670 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.035533    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-online-tls\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.035670 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.035580    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-feast-data\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.035670 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.035595    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-registry-tls\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.035670 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.035624    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c6d6c\" (UniqueName: \"kubernetes.io/projected/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-kube-api-access-c6d6c\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.035885 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:50:42.035728    2551 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found
May 26 02:50:42.035885 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:50:42.035761    2551 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found
May 26 02:50:42.035885 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:50:42.035799    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-online-tls podName:7b86dde0-2562-4e29-a156-d8fe3e99b0f5 nodeName:}" failed. No retries permitted until 2026-05-26 02:50:42.535767329 +0000 UTC m=+880.092555990 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-online-tls") pod "feast-credit-scoring-5b478b98d4-pfskl" (UID: "7b86dde0-2562-4e29-a156-d8fe3e99b0f5") : secret "feast-credit-scoring-online-tls" not found
May 26 02:50:42.035885 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:50:42.035834    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-registry-tls podName:7b86dde0-2562-4e29-a156-d8fe3e99b0f5 nodeName:}" failed. No retries permitted until 2026-05-26 02:50:42.535816741 +0000 UTC m=+880.092605396 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-registry-tls") pod "feast-credit-scoring-5b478b98d4-pfskl" (UID: "7b86dde0-2562-4e29-a156-d8fe3e99b0f5") : secret "feast-credit-scoring-registry-tls" not found
May 26 02:50:42.036120 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.036100    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-feast-data\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.038124 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.038099    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-offline-tls\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.043879 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.043859    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6d6c\" (UniqueName: \"kubernetes.io/projected/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-kube-api-access-c6d6c\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.540988 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.540955    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-online-tls\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.541151 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.541024    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-registry-tls\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.543425 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.543407    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-registry-tls\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.543508 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.543485    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-online-tls\") pod \"feast-credit-scoring-5b478b98d4-pfskl\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") " pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.810438 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.810411    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:42.940246 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:42.940219    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"]
May 26 02:50:42.941588 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:50:42.941561    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod7b86dde0_2562_4e29_a156_d8fe3e99b0f5.slice/crio-59b3aacfda0694747477c57e282c8f9059e590659edcdf63c9c1376bcb496150 WatchSource:0}: Error finding container 59b3aacfda0694747477c57e282c8f9059e590659edcdf63c9c1376bcb496150: Status 404 returned error can't find the container with id 59b3aacfda0694747477c57e282c8f9059e590659edcdf63c9c1376bcb496150
May 26 02:50:43.925600 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:43.925567    2551 generic.go:358] "Generic (PLEG): container finished" podID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerID="a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7" exitCode=0
May 26 02:50:43.925735 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:43.925652    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" event={"ID":"7b86dde0-2562-4e29-a156-d8fe3e99b0f5","Type":"ContainerDied","Data":"a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7"}
May 26 02:50:43.925735 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:43.925695    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" event={"ID":"7b86dde0-2562-4e29-a156-d8fe3e99b0f5","Type":"ContainerStarted","Data":"59b3aacfda0694747477c57e282c8f9059e590659edcdf63c9c1376bcb496150"}
May 26 02:50:44.931081 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:44.931052    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" event={"ID":"7b86dde0-2562-4e29-a156-d8fe3e99b0f5","Type":"ContainerStarted","Data":"e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b"}
May 26 02:50:50.951248 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:50.951213    2551 generic.go:358] "Generic (PLEG): container finished" podID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerID="e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b" exitCode=0
May 26 02:50:50.951655 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:50.951261    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" event={"ID":"7b86dde0-2562-4e29-a156-d8fe3e99b0f5","Type":"ContainerDied","Data":"e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b"}
May 26 02:50:51.958795 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:51.958754    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" event={"ID":"7b86dde0-2562-4e29-a156-d8fe3e99b0f5","Type":"ContainerStarted","Data":"f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694"}
May 26 02:50:51.959205 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:51.958811    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" event={"ID":"7b86dde0-2562-4e29-a156-d8fe3e99b0f5","Type":"ContainerStarted","Data":"ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595"}
May 26 02:50:51.959205 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:51.958828    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" event={"ID":"7b86dde0-2562-4e29-a156-d8fe3e99b0f5","Type":"ContainerStarted","Data":"d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c"}
May 26 02:50:51.984465 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:51.984371    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" podStartSLOduration=10.984350555 podStartE2EDuration="10.984350555s" podCreationTimestamp="2026-05-26 02:50:41 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:50:51.982408862 +0000 UTC m=+889.539197532" watchObservedRunningTime="2026-05-26 02:50:51.984350555 +0000 UTC m=+889.541139226"
May 26 02:50:52.810702 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:52.810661    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:52.810915 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:52.810718    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:52.810915 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:52.810733    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:54.811637 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:54.811598    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:54.812133 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:54.811649    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:54.812133 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:54.811662    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:54.813400 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:54.813365    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 26 02:50:54.813400 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:54.813379    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.32:8816: connect: connection refused"
May 26 02:50:54.813608 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:54.813464    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 26 02:50:57.811294 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:57.811240    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 26 02:50:57.811863 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:57.811839    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:57.812854 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:57.812829    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:57.816606 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:57.816583    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:50:57.982711 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:50:57.982679    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:51:00.810740 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:51:00.810689    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 26 02:51:02.964286 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:51:02.964260    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 02:51:02.969349 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:51:02.969327    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 02:51:02.972170 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:51:02.972149    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 02:51:02.973794 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:51:02.973776    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 02:51:03.811604 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:51:03.811566    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:51:03.812419 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:51:03.812400    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:52:05.456079 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.456037    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"]
May 26 02:52:05.456733 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.456497    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="registry" containerID="cri-o://d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c" gracePeriod=30
May 26 02:52:05.456733 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.456594    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="offline" containerID="cri-o://f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694" gracePeriod=30
May 26 02:52:05.457466 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.456580    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="online" containerID="cri-o://ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595" gracePeriod=30
May 26 02:52:05.471786 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.471761    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-ms7s5"]
May 26 02:52:05.471982 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.471962    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/postgres-7c7656cc6d-ms7s5" podUID="86bf3be7-b7e7-4a26-aa51-f6591025497a" containerName="postgres" containerID="cri-o://d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2" gracePeriod=30
May 26 02:52:05.716363 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.716306    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-ms7s5"
May 26 02:52:05.857636 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.857599    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/86bf3be7-b7e7-4a26-aa51-f6591025497a-postgresdata\") pod \"86bf3be7-b7e7-4a26-aa51-f6591025497a\" (UID: \"86bf3be7-b7e7-4a26-aa51-f6591025497a\") "
May 26 02:52:05.857802 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.857708    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-fgdqk\" (UniqueName: \"kubernetes.io/projected/86bf3be7-b7e7-4a26-aa51-f6591025497a-kube-api-access-fgdqk\") pod \"86bf3be7-b7e7-4a26-aa51-f6591025497a\" (UID: \"86bf3be7-b7e7-4a26-aa51-f6591025497a\") "
May 26 02:52:05.859799 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.859767    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/86bf3be7-b7e7-4a26-aa51-f6591025497a-kube-api-access-fgdqk" (OuterVolumeSpecName: "kube-api-access-fgdqk") pod "86bf3be7-b7e7-4a26-aa51-f6591025497a" (UID: "86bf3be7-b7e7-4a26-aa51-f6591025497a"). InnerVolumeSpecName "kube-api-access-fgdqk". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:52:05.887654 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.887615    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/86bf3be7-b7e7-4a26-aa51-f6591025497a-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "86bf3be7-b7e7-4a26-aa51-f6591025497a" (UID: "86bf3be7-b7e7-4a26-aa51-f6591025497a"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 02:52:05.959010 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.958973    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-fgdqk\" (UniqueName: \"kubernetes.io/projected/86bf3be7-b7e7-4a26-aa51-f6591025497a-kube-api-access-fgdqk\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:52:05.959010 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:05.959007    2551 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/86bf3be7-b7e7-4a26-aa51-f6591025497a-postgresdata\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:52:06.184412 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:06.184377    2551 generic.go:358] "Generic (PLEG): container finished" podID="86bf3be7-b7e7-4a26-aa51-f6591025497a" containerID="d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2" exitCode=0
May 26 02:52:06.184559 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:06.184441    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-ms7s5"
May 26 02:52:06.184559 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:06.184456    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-ms7s5" event={"ID":"86bf3be7-b7e7-4a26-aa51-f6591025497a","Type":"ContainerDied","Data":"d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2"}
May 26 02:52:06.184559 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:06.184492    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-ms7s5" event={"ID":"86bf3be7-b7e7-4a26-aa51-f6591025497a","Type":"ContainerDied","Data":"835548df1cdca918c13a8ae511d49bd63ab757a0a50b45aeb44b1a1c2e449e44"}
May 26 02:52:06.184559 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:06.184509    2551 scope.go:117] "RemoveContainer" containerID="d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2"
May 26 02:52:06.187022 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:06.187000    2551 generic.go:358] "Generic (PLEG): container finished" podID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerID="ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595" exitCode=0
May 26 02:52:06.187146 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:06.187075    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" event={"ID":"7b86dde0-2562-4e29-a156-d8fe3e99b0f5","Type":"ContainerDied","Data":"ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595"}
May 26 02:52:06.191693 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:06.191666    2551 scope.go:117] "RemoveContainer" containerID="d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2"
May 26 02:52:06.191980 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:52:06.191954    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2\": container with ID starting with d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2 not found: ID does not exist" containerID="d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2"
May 26 02:52:06.192064 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:06.191990    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2"} err="failed to get container status \"d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2\": rpc error: code = NotFound desc = could not find container \"d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2\": container with ID starting with d4cc2a77dde880d7208e5687c5a396826efe4e8ae468f705ceaaebdf94cc93d2 not found: ID does not exist"
May 26 02:52:06.204307 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:06.204287    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-ms7s5"]
May 26 02:52:06.207669 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:06.207648    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-ms7s5"]
May 26 02:52:07.045847 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:07.045815    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="86bf3be7-b7e7-4a26-aa51-f6591025497a" path="/var/lib/kubelet/pods/86bf3be7-b7e7-4a26-aa51-f6591025497a/volumes"
May 26 02:52:07.980083 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:07.980041    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 26 02:52:17.980308 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:17.980209    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 26 02:52:27.979918 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:27.979862    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 26 02:52:27.980364 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:27.980030    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:52:36.115993 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.115968    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:52:36.208584 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.208551    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-feast-data\") pod \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") "
May 26 02:52:36.208584 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.208588    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6d6c\" (UniqueName: \"kubernetes.io/projected/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-kube-api-access-c6d6c\") pod \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") "
May 26 02:52:36.208787 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.208607    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-offline-tls\") pod \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") "
May 26 02:52:36.208787 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.208624    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-online-tls\") pod \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") "
May 26 02:52:36.208787 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.208672    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-registry-tls\") pod \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\" (UID: \"7b86dde0-2562-4e29-a156-d8fe3e99b0f5\") "
May 26 02:52:36.210918 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.210864    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "7b86dde0-2562-4e29-a156-d8fe3e99b0f5" (UID: "7b86dde0-2562-4e29-a156-d8fe3e99b0f5"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:52:36.210918 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.210874    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-kube-api-access-c6d6c" (OuterVolumeSpecName: "kube-api-access-c6d6c") pod "7b86dde0-2562-4e29-a156-d8fe3e99b0f5" (UID: "7b86dde0-2562-4e29-a156-d8fe3e99b0f5"). InnerVolumeSpecName "kube-api-access-c6d6c". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:52:36.211073 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.210980    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-online-tls" (OuterVolumeSpecName: "online-tls") pod "7b86dde0-2562-4e29-a156-d8fe3e99b0f5" (UID: "7b86dde0-2562-4e29-a156-d8fe3e99b0f5"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:52:36.211111 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.211072    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "7b86dde0-2562-4e29-a156-d8fe3e99b0f5" (UID: "7b86dde0-2562-4e29-a156-d8fe3e99b0f5"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:52:36.217596 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.217569    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-feast-data" (OuterVolumeSpecName: "feast-data") pod "7b86dde0-2562-4e29-a156-d8fe3e99b0f5" (UID: "7b86dde0-2562-4e29-a156-d8fe3e99b0f5"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 02:52:36.276793 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.276736    2551 generic.go:358] "Generic (PLEG): container finished" podID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerID="f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694" exitCode=137
May 26 02:52:36.276793 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.276757    2551 generic.go:358] "Generic (PLEG): container finished" podID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerID="d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c" exitCode=137
May 26 02:52:36.276928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.276797    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" event={"ID":"7b86dde0-2562-4e29-a156-d8fe3e99b0f5","Type":"ContainerDied","Data":"f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694"}
May 26 02:52:36.276928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.276823    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" event={"ID":"7b86dde0-2562-4e29-a156-d8fe3e99b0f5","Type":"ContainerDied","Data":"d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c"}
May 26 02:52:36.276928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.276830    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"
May 26 02:52:36.276928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.276843    2551 scope.go:117] "RemoveContainer" containerID="f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694"
May 26 02:52:36.276928 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.276833    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl" event={"ID":"7b86dde0-2562-4e29-a156-d8fe3e99b0f5","Type":"ContainerDied","Data":"59b3aacfda0694747477c57e282c8f9059e590659edcdf63c9c1376bcb496150"}
May 26 02:52:36.285020 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.284865    2551 scope.go:117] "RemoveContainer" containerID="ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595"
May 26 02:52:36.292216 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.292200    2551 scope.go:117] "RemoveContainer" containerID="d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c"
May 26 02:52:36.298982 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.298958    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"]
May 26 02:52:36.299400 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.299382    2551 scope.go:117] "RemoveContainer" containerID="e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b"
May 26 02:52:36.302941 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.302918    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-5b478b98d4-pfskl"]
May 26 02:52:36.307095 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.307078    2551 scope.go:117] "RemoveContainer" containerID="a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7"
May 26 02:52:36.310238 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.310218    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:52:36.310238 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.310239    2551 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-feast-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:52:36.310352 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.310248    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6d6c\" (UniqueName: \"kubernetes.io/projected/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-kube-api-access-c6d6c\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:52:36.310352 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.310259    2551 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-offline-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:52:36.310352 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.310268    2551 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/7b86dde0-2562-4e29-a156-d8fe3e99b0f5-online-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:52:36.313590 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.313573    2551 scope.go:117] "RemoveContainer" containerID="f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694"
May 26 02:52:36.313832 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:52:36.313806    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694\": container with ID starting with f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694 not found: ID does not exist" containerID="f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694"
May 26 02:52:36.313870 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.313842    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694"} err="failed to get container status \"f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694\": rpc error: code = NotFound desc = could not find container \"f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694\": container with ID starting with f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694 not found: ID does not exist"
May 26 02:52:36.313870 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.313859    2551 scope.go:117] "RemoveContainer" containerID="ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595"
May 26 02:52:36.314125 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:52:36.314109    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595\": container with ID starting with ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595 not found: ID does not exist" containerID="ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595"
May 26 02:52:36.314175 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.314128    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595"} err="failed to get container status \"ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595\": rpc error: code = NotFound desc = could not find container \"ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595\": container with ID starting with ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595 not found: ID does not exist"
May 26 02:52:36.314175 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.314144    2551 scope.go:117] "RemoveContainer" containerID="d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c"
May 26 02:52:36.314353 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:52:36.314334    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c\": container with ID starting with d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c not found: ID does not exist" containerID="d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c"
May 26 02:52:36.314414 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.314364    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c"} err="failed to get container status \"d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c\": rpc error: code = NotFound desc = could not find container \"d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c\": container with ID starting with d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c not found: ID does not exist"
May 26 02:52:36.314414 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.314387    2551 scope.go:117] "RemoveContainer" containerID="e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b"
May 26 02:52:36.314625 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:52:36.314608    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b\": container with ID starting with e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b not found: ID does not exist" containerID="e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b"
May 26 02:52:36.314665 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.314630    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b"} err="failed to get container status \"e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b\": rpc error: code = NotFound desc = could not find container \"e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b\": container with ID starting with e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b not found: ID does not exist"
May 26 02:52:36.314665 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.314645    2551 scope.go:117] "RemoveContainer" containerID="a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7"
May 26 02:52:36.314834 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:52:36.314818    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7\": container with ID starting with a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7 not found: ID does not exist" containerID="a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7"
May 26 02:52:36.314891 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.314842    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7"} err="failed to get container status \"a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7\": rpc error: code = NotFound desc = could not find container \"a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7\": container with ID starting with a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7 not found: ID does not exist"
May 26 02:52:36.314891 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.314864    2551 scope.go:117] "RemoveContainer" containerID="f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694"
May 26 02:52:36.315114 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.315092    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694"} err="failed to get container status \"f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694\": rpc error: code = NotFound desc = could not find container \"f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694\": container with ID starting with f03264a8a65e4a775464b8c1164a5036baca1ef71cfb5d63b905a7b81ab7b694 not found: ID does not exist"
May 26 02:52:36.315162 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.315115    2551 scope.go:117] "RemoveContainer" containerID="ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595"
May 26 02:52:36.315318 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.315298    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595"} err="failed to get container status \"ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595\": rpc error: code = NotFound desc = could not find container \"ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595\": container with ID starting with ece53ea947b29e7d2a8a6df7a79e311ebffb53358f5e980e1acf742577ce8595 not found: ID does not exist"
May 26 02:52:36.315382 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.315319    2551 scope.go:117] "RemoveContainer" containerID="d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c"
May 26 02:52:36.315546 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.315529    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c"} err="failed to get container status \"d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c\": rpc error: code = NotFound desc = could not find container \"d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c\": container with ID starting with d8a422dec659c811ce49c36775dcb94d18d3487d16f1cf888667816abe61cf9c not found: ID does not exist"
May 26 02:52:36.315598 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.315546    2551 scope.go:117] "RemoveContainer" containerID="e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b"
May 26 02:52:36.315751 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.315733    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b"} err="failed to get container status \"e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b\": rpc error: code = NotFound desc = could not find container \"e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b\": container with ID starting with e4ae43f82c0e22b5c9aacffb6ad3fc2a823b88f922b9dbf3291cbfa8e709a98b not found: ID does not exist"
May 26 02:52:36.315802 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.315753    2551 scope.go:117] "RemoveContainer" containerID="a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7"
May 26 02:52:36.315973 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:36.315957    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7"} err="failed to get container status \"a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7\": rpc error: code = NotFound desc = could not find container \"a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7\": container with ID starting with a7e03651acf26bfc913b1230abf2c85d3b1338900c206e7da2b94e7b114716a7 not found: ID does not exist"
May 26 02:52:37.044620 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:52:37.044586    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" path="/var/lib/kubelet/pods/7b86dde0-2562-4e29-a156-d8fe3e99b0f5/volumes"
May 26 02:54:14.177701 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.177618    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-854544c579-g52h7"]
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178020    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="offline"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178038    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="offline"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178049    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="86bf3be7-b7e7-4a26-aa51-f6591025497a" containerName="postgres"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178056    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="86bf3be7-b7e7-4a26-aa51-f6591025497a" containerName="postgres"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178069    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="feast-init"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178077    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="feast-init"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178087    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="registry"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178094    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="registry"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178122    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="feast-apply"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178130    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="feast-apply"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178141    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="online"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178148    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="online"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178212    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="offline"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178223    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="online"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178234    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="86bf3be7-b7e7-4a26-aa51-f6591025497a" containerName="postgres"
May 26 02:54:14.178251 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.178242    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="7b86dde0-2562-4e29-a156-d8fe3e99b0f5" containerName="registry"
May 26 02:54:14.181278 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.181257    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-g52h7"
May 26 02:54:14.183858 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.183835    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
May 26 02:54:14.183971 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.183882    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
May 26 02:54:14.184675 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.184656    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-krjr9\""
May 26 02:54:14.207229 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.207206    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-g52h7"]
May 26 02:54:14.211839 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.211820    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-f7jfc\" (UniqueName: \"kubernetes.io/projected/eea90396-49c4-4c78-affc-b747df45631c-kube-api-access-f7jfc\") pod \"redis-854544c579-g52h7\" (UID: \"eea90396-49c4-4c78-affc-b747df45631c\") " pod="test-ns-feast-rest/redis-854544c579-g52h7"
May 26 02:54:14.312737 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.312683    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-f7jfc\" (UniqueName: \"kubernetes.io/projected/eea90396-49c4-4c78-affc-b747df45631c-kube-api-access-f7jfc\") pod \"redis-854544c579-g52h7\" (UID: \"eea90396-49c4-4c78-affc-b747df45631c\") " pod="test-ns-feast-rest/redis-854544c579-g52h7"
May 26 02:54:14.320531 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.320508    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-f7jfc\" (UniqueName: \"kubernetes.io/projected/eea90396-49c4-4c78-affc-b747df45631c-kube-api-access-f7jfc\") pod \"redis-854544c579-g52h7\" (UID: \"eea90396-49c4-4c78-affc-b747df45631c\") " pod="test-ns-feast-rest/redis-854544c579-g52h7"
May 26 02:54:14.497629 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.497546    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-g52h7"
May 26 02:54:14.622779 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.622752    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-g52h7"]
May 26 02:54:14.624970 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:54:14.624940    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podeea90396_49c4_4c78_affc_b747df45631c.slice/crio-b534c442c3225d4dbd2021ef207aeb7f579565e72c831776eb224eb615144136 WatchSource:0}: Error finding container b534c442c3225d4dbd2021ef207aeb7f579565e72c831776eb224eb615144136: Status 404 returned error can't find the container with id b534c442c3225d4dbd2021ef207aeb7f579565e72c831776eb224eb615144136
May 26 02:54:14.626583 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:14.626566    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 26 02:54:15.569647 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:15.569609    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-g52h7" event={"ID":"eea90396-49c4-4c78-affc-b747df45631c","Type":"ContainerStarted","Data":"b534c442c3225d4dbd2021ef207aeb7f579565e72c831776eb224eb615144136"}
May 26 02:54:16.573609 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:16.573570    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-g52h7" event={"ID":"eea90396-49c4-4c78-affc-b747df45631c","Type":"ContainerStarted","Data":"be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228"}
May 26 02:54:16.588259 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:16.588214    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/redis-854544c579-g52h7" podStartSLOduration=1.243115749 podStartE2EDuration="2.588198663s" podCreationTimestamp="2026-05-26 02:54:14 +0000 UTC" firstStartedPulling="2026-05-26 02:54:14.626720451 +0000 UTC m=+1092.183509098" lastFinishedPulling="2026-05-26 02:54:15.971803353 +0000 UTC m=+1093.528592012" observedRunningTime="2026-05-26 02:54:16.586631207 +0000 UTC m=+1094.143419876" watchObservedRunningTime="2026-05-26 02:54:16.588198663 +0000 UTC m=+1094.144987362"
May 26 02:54:19.605958 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:19.605923    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"]
May 26 02:54:19.609132 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:19.609113    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"
May 26 02:54:19.611216 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:19.611197    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\""
May 26 02:54:19.614931 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:19.614891    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"]
May 26 02:54:19.659500 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:19.659472    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/95d80b8d-f5fd-476b-b52b-c84fec865b68-postgresdata\") pod \"postgres-7c7656cc6d-2n2n6\" (UID: \"95d80b8d-f5fd-476b-b52b-c84fec865b68\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"
May 26 02:54:19.659606 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:19.659506    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jcsnd\" (UniqueName: \"kubernetes.io/projected/95d80b8d-f5fd-476b-b52b-c84fec865b68-kube-api-access-jcsnd\") pod \"postgres-7c7656cc6d-2n2n6\" (UID: \"95d80b8d-f5fd-476b-b52b-c84fec865b68\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"
May 26 02:54:19.760765 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:19.760734    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jcsnd\" (UniqueName: \"kubernetes.io/projected/95d80b8d-f5fd-476b-b52b-c84fec865b68-kube-api-access-jcsnd\") pod \"postgres-7c7656cc6d-2n2n6\" (UID: \"95d80b8d-f5fd-476b-b52b-c84fec865b68\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"
May 26 02:54:19.760892 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:19.760838    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/95d80b8d-f5fd-476b-b52b-c84fec865b68-postgresdata\") pod \"postgres-7c7656cc6d-2n2n6\" (UID: \"95d80b8d-f5fd-476b-b52b-c84fec865b68\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"
May 26 02:54:19.761195 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:19.761178    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/95d80b8d-f5fd-476b-b52b-c84fec865b68-postgresdata\") pod \"postgres-7c7656cc6d-2n2n6\" (UID: \"95d80b8d-f5fd-476b-b52b-c84fec865b68\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"
May 26 02:54:19.768895 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:19.768872    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jcsnd\" (UniqueName: \"kubernetes.io/projected/95d80b8d-f5fd-476b-b52b-c84fec865b68-kube-api-access-jcsnd\") pod \"postgres-7c7656cc6d-2n2n6\" (UID: \"95d80b8d-f5fd-476b-b52b-c84fec865b68\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"
May 26 02:54:19.920682 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:19.920597    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"
May 26 02:54:20.038822 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:20.038781    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"]
May 26 02:54:20.042050 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:54:20.042019    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod95d80b8d_f5fd_476b_b52b_c84fec865b68.slice/crio-b7ec1c92eed6a4449160410ad6f7f529fe73220b43eaa02e2bf06ee89929ca0c WatchSource:0}: Error finding container b7ec1c92eed6a4449160410ad6f7f529fe73220b43eaa02e2bf06ee89929ca0c: Status 404 returned error can't find the container with id b7ec1c92eed6a4449160410ad6f7f529fe73220b43eaa02e2bf06ee89929ca0c
May 26 02:54:20.586513 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:20.586468    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6" event={"ID":"95d80b8d-f5fd-476b-b52b-c84fec865b68","Type":"ContainerStarted","Data":"a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e"}
May 26 02:54:20.586513 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:20.586507    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6" event={"ID":"95d80b8d-f5fd-476b-b52b-c84fec865b68","Type":"ContainerStarted","Data":"b7ec1c92eed6a4449160410ad6f7f529fe73220b43eaa02e2bf06ee89929ca0c"}
May 26 02:54:20.601426 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:20.601371    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6" podStartSLOduration=1.60135419 podStartE2EDuration="1.60135419s" podCreationTimestamp="2026-05-26 02:54:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:54:20.600275489 +0000 UTC m=+1098.157064171" watchObservedRunningTime="2026-05-26 02:54:20.60135419 +0000 UTC m=+1098.158142862"
May 26 02:54:25.232743 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.232702    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"]
May 26 02:54:25.236760 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.236740    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.238886 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.238860    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\""
May 26 02:54:25.239027 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.238989    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
May 26 02:54:25.239098 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.239044    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\""
May 26 02:54:25.239098 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.239064    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-q5b69\""
May 26 02:54:25.246975 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.246950    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"]
May 26 02:54:25.310268 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.310228    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qqwcx\" (UniqueName: \"kubernetes.io/projected/d3152c75-dc45-4de9-b0fb-880dcd9d9181-kube-api-access-qqwcx\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.310268 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.310264    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-registry-tls\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.310500 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.310284    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-online-tls\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.310500 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.310407    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d3152c75-dc45-4de9-b0fb-880dcd9d9181-feast-data\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.411534 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.411490    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qqwcx\" (UniqueName: \"kubernetes.io/projected/d3152c75-dc45-4de9-b0fb-880dcd9d9181-kube-api-access-qqwcx\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.411534 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.411531    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-registry-tls\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.411788 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.411549    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-online-tls\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.411788 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.411595    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d3152c75-dc45-4de9-b0fb-880dcd9d9181-feast-data\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.411788 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:54:25.411637    2551 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-registry-tls: secret "feast-credit-scoring-registry-tls" not found
May 26 02:54:25.411788 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:54:25.411724    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-registry-tls podName:d3152c75-dc45-4de9-b0fb-880dcd9d9181 nodeName:}" failed. No retries permitted until 2026-05-26 02:54:25.911706745 +0000 UTC m=+1103.468495392 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-registry-tls") pod "feast-credit-scoring-6cc7dbc64c-9j66m" (UID: "d3152c75-dc45-4de9-b0fb-880dcd9d9181") : secret "feast-credit-scoring-registry-tls" not found
May 26 02:54:25.412034 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.412015    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d3152c75-dc45-4de9-b0fb-880dcd9d9181-feast-data\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.414082 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.414064    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-online-tls\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.419894 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.419864    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qqwcx\" (UniqueName: \"kubernetes.io/projected/d3152c75-dc45-4de9-b0fb-880dcd9d9181-kube-api-access-qqwcx\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.917028 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.916987    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-registry-tls\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:25.919429 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:25.919398    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-registry-tls\") pod \"feast-credit-scoring-6cc7dbc64c-9j66m\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") " pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:26.148872 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:26.148831    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:26.280964 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:26.280939    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"]
May 26 02:54:26.283562 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:54:26.283533    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podd3152c75_dc45_4de9_b0fb_880dcd9d9181.slice/crio-c3e930b01eff33749b3a33dc7beb60e4218e37e472b1b7262c78e41faa84084b WatchSource:0}: Error finding container c3e930b01eff33749b3a33dc7beb60e4218e37e472b1b7262c78e41faa84084b: Status 404 returned error can't find the container with id c3e930b01eff33749b3a33dc7beb60e4218e37e472b1b7262c78e41faa84084b
May 26 02:54:26.608632 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:26.608597    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" event={"ID":"d3152c75-dc45-4de9-b0fb-880dcd9d9181","Type":"ContainerStarted","Data":"ed0399c49d8cb111b0332325a8df136fc8171e981a6b4a448310bf8287eb9676"}
May 26 02:54:26.608632 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:26.608636    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" event={"ID":"d3152c75-dc45-4de9-b0fb-880dcd9d9181","Type":"ContainerStarted","Data":"c3e930b01eff33749b3a33dc7beb60e4218e37e472b1b7262c78e41faa84084b"}
May 26 02:54:27.612583 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:27.612547    2551 generic.go:358] "Generic (PLEG): container finished" podID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerID="ed0399c49d8cb111b0332325a8df136fc8171e981a6b4a448310bf8287eb9676" exitCode=0
May 26 02:54:27.612583 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:27.612583    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" event={"ID":"d3152c75-dc45-4de9-b0fb-880dcd9d9181","Type":"ContainerDied","Data":"ed0399c49d8cb111b0332325a8df136fc8171e981a6b4a448310bf8287eb9676"}
May 26 02:54:28.617959 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:28.617925    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" event={"ID":"d3152c75-dc45-4de9-b0fb-880dcd9d9181","Type":"ContainerStarted","Data":"f87f97a00e7b9ecc76b76e7bc288f03c15ba8dc946e09a8e9ef74b764a97f308"}
May 26 02:54:35.638879 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:35.638846    2551 generic.go:358] "Generic (PLEG): container finished" podID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerID="f87f97a00e7b9ecc76b76e7bc288f03c15ba8dc946e09a8e9ef74b764a97f308" exitCode=0
May 26 02:54:35.639261 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:35.638925    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" event={"ID":"d3152c75-dc45-4de9-b0fb-880dcd9d9181","Type":"ContainerDied","Data":"f87f97a00e7b9ecc76b76e7bc288f03c15ba8dc946e09a8e9ef74b764a97f308"}
May 26 02:54:36.644539 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:36.644500    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" event={"ID":"d3152c75-dc45-4de9-b0fb-880dcd9d9181","Type":"ContainerStarted","Data":"d8cee060888a2d1a0a81e81e4153effe4d3b067b53425ddde94168e461a77ef9"}
May 26 02:54:36.644943 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:36.644553    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" event={"ID":"d3152c75-dc45-4de9-b0fb-880dcd9d9181","Type":"ContainerStarted","Data":"c2f23f9d1235846105d39bf6b7ab01cc25f6f6e566f754c54352bbdcd9ddc567"}
May 26 02:54:36.667658 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:36.667607    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" podStartSLOduration=11.667589417 podStartE2EDuration="11.667589417s" podCreationTimestamp="2026-05-26 02:54:25 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:54:36.665001799 +0000 UTC m=+1114.221790468" watchObservedRunningTime="2026-05-26 02:54:36.667589417 +0000 UTC m=+1114.224378087"
May 26 02:54:38.149859 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:38.149823    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:38.149859 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:38.149870    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:38.151543 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:38.151517    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
May 26 02:54:38.151632 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:38.151535    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 26 02:54:41.149673 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:41.149626    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
May 26 02:54:41.728068 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:41.728034    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:41.728523 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:41.728502    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:41.731297 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:41.731273    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:44.149884 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:44.149839    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.35:6571: connect: connection refused"
May 26 02:54:46.149528 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:46.149486    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:47.149814 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:47.149784    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:47.150535 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:47.150518    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:54:51.068952 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.068919    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"]
May 26 02:54:51.074159 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.074127    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.076989 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.076963    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-qx4q9\""
May 26 02:54:51.077244 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.077220    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
May 26 02:54:51.077336 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.077247    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\""
May 26 02:54:51.077336 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.077261    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\""
May 26 02:54:51.083545 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.083491    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"]
May 26 02:54:51.142118 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.142095    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-online-tls\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.142244 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.142152    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-pf6sw\" (UniqueName: \"kubernetes.io/projected/bbd974f9-a60d-4707-963b-40715d92e3bb-kube-api-access-pf6sw\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.142244 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.142222    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-registry-tls\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.142318 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.142267    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd974f9-a60d-4707-963b-40715d92e3bb-feast-data\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.242778 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.242751    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-online-tls\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.242934 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.242809    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-pf6sw\" (UniqueName: \"kubernetes.io/projected/bbd974f9-a60d-4707-963b-40715d92e3bb-kube-api-access-pf6sw\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.242934 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.242837    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-registry-tls\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.242934 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.242866    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd974f9-a60d-4707-963b-40715d92e3bb-feast-data\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.243048 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:54:51.242979    2551 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-registry-tls: secret "feast-driver-ranking-registry-tls" not found
May 26 02:54:51.243048 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:54:51.243041    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-registry-tls podName:bbd974f9-a60d-4707-963b-40715d92e3bb nodeName:}" failed. No retries permitted until 2026-05-26 02:54:51.743022697 +0000 UTC m=+1129.299811366 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-registry-tls") pod "feast-driver-ranking-7df4f69496-dclh8" (UID: "bbd974f9-a60d-4707-963b-40715d92e3bb") : secret "feast-driver-ranking-registry-tls" not found
May 26 02:54:51.243264 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.243235    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd974f9-a60d-4707-963b-40715d92e3bb-feast-data\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.245414 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.245388    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-online-tls\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.251220 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.251120    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-pf6sw\" (UniqueName: \"kubernetes.io/projected/bbd974f9-a60d-4707-963b-40715d92e3bb-kube-api-access-pf6sw\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.746106 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.746073    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-registry-tls\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.748303 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.748283    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-registry-tls\") pod \"feast-driver-ranking-7df4f69496-dclh8\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") " pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:51.987449 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:51.987412    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:54:52.119191 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:52.119163    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"]
May 26 02:54:52.120446 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:54:52.120421    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podbbd974f9_a60d_4707_963b_40715d92e3bb.slice/crio-ecbe1c02f44b86ec999b5fd43e085731cb786de152dff9ac432534db6871f69b WatchSource:0}: Error finding container ecbe1c02f44b86ec999b5fd43e085731cb786de152dff9ac432534db6871f69b: Status 404 returned error can't find the container with id ecbe1c02f44b86ec999b5fd43e085731cb786de152dff9ac432534db6871f69b
May 26 02:54:52.700236 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:52.700196    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" event={"ID":"bbd974f9-a60d-4707-963b-40715d92e3bb","Type":"ContainerStarted","Data":"b50844a60d9a1d670db1194a06f653f68db7861691cf08ce6154ea2951d6e3fa"}
May 26 02:54:52.700236 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:52.700235    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" event={"ID":"bbd974f9-a60d-4707-963b-40715d92e3bb","Type":"ContainerStarted","Data":"ecbe1c02f44b86ec999b5fd43e085731cb786de152dff9ac432534db6871f69b"}
May 26 02:54:56.715469 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:56.715432    2551 generic.go:358] "Generic (PLEG): container finished" podID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerID="b50844a60d9a1d670db1194a06f653f68db7861691cf08ce6154ea2951d6e3fa" exitCode=0
May 26 02:54:56.715861 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:56.715502    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" event={"ID":"bbd974f9-a60d-4707-963b-40715d92e3bb","Type":"ContainerDied","Data":"b50844a60d9a1d670db1194a06f653f68db7861691cf08ce6154ea2951d6e3fa"}
May 26 02:54:57.720546 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:54:57.720517    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" event={"ID":"bbd974f9-a60d-4707-963b-40715d92e3bb","Type":"ContainerStarted","Data":"eb9bea3d0678aab8558a49eb07c9bf82b70194e4a6fcfc3f528e1bad280a0b9a"}
May 26 02:55:03.740232 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:03.740197    2551 generic.go:358] "Generic (PLEG): container finished" podID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerID="eb9bea3d0678aab8558a49eb07c9bf82b70194e4a6fcfc3f528e1bad280a0b9a" exitCode=0
May 26 02:55:03.740602 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:03.740253    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" event={"ID":"bbd974f9-a60d-4707-963b-40715d92e3bb","Type":"ContainerDied","Data":"eb9bea3d0678aab8558a49eb07c9bf82b70194e4a6fcfc3f528e1bad280a0b9a"}
May 26 02:55:03.743847 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:03.743828    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
May 26 02:55:04.746797 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:04.746761    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" event={"ID":"bbd974f9-a60d-4707-963b-40715d92e3bb","Type":"ContainerStarted","Data":"574b525daf01edaf403dbb7869c1d4e90c30fc23d41c00776be001d815a7b4ad"}
May 26 02:55:04.747289 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:04.746807    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" event={"ID":"bbd974f9-a60d-4707-963b-40715d92e3bb","Type":"ContainerStarted","Data":"6a0dc69f7e92b8f76c817a25e959a47ccf01394b598cbb74276c6ecad604051e"}
May 26 02:55:04.768726 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:04.768676    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" podStartSLOduration=13.768662435 podStartE2EDuration="13.768662435s" podCreationTimestamp="2026-05-26 02:54:51 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:55:04.765857062 +0000 UTC m=+1142.322645731" watchObservedRunningTime="2026-05-26 02:55:04.768662435 +0000 UTC m=+1142.325451104"
May 26 02:55:06.988453 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:06.988408    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:55:06.988940 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:06.988466    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:55:06.990200 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:06.990158    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused"
May 26 02:55:06.990321 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:06.990208    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 26 02:55:09.988423 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:09.988386    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused"
May 26 02:55:09.993697 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:09.993668    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:55:09.994119 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:09.994098    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:55:09.996886 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:09.996863    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:55:11.988478 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:11.988429    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:55:12.988249 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:12.988200    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused"
May 26 02:55:15.988608 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:15.988526    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:55:15.989202 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:55:15.989185    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:56:02.992746 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:02.992715    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 02:56:02.995412 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:02.995393    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 02:56:02.997309 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:02.997291    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 02:56:03.000065 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:03.000047    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 02:56:29.725108 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:29.725071    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"]
May 26 02:56:29.725560 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:29.725314    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc" podUID="8fa41aea-c330-460d-8a3f-118ec9449f26" containerName="manager" containerID="cri-o://7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2" gracePeriod=10
May 26 02:56:29.957167 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:29.957132    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"]
May 26 02:56:29.957607 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:29.957557    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="registry" containerID="cri-o://c2f23f9d1235846105d39bf6b7ab01cc25f6f6e566f754c54352bbdcd9ddc567" gracePeriod=30
May 26 02:56:29.957768 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:29.957618    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="online" containerID="cri-o://d8cee060888a2d1a0a81e81e4153effe4d3b067b53425ddde94168e461a77ef9" gracePeriod=30
May 26 02:56:29.962985 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:29.962959    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"]
May 26 02:56:29.963396 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:29.963353    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="registry" containerID="cri-o://6a0dc69f7e92b8f76c817a25e959a47ccf01394b598cbb74276c6ecad604051e" gracePeriod=30
May 26 02:56:29.963600 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:29.963372    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="online" containerID="cri-o://574b525daf01edaf403dbb7869c1d4e90c30fc23d41c00776be001d815a7b4ad" gracePeriod=30
May 26 02:56:30.015234 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.015213    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"
May 26 02:56:30.036683 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.036650    2551 generic.go:358] "Generic (PLEG): container finished" podID="8fa41aea-c330-460d-8a3f-118ec9449f26" containerID="7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2" exitCode=0
May 26 02:56:30.036840 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.036693    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc" event={"ID":"8fa41aea-c330-460d-8a3f-118ec9449f26","Type":"ContainerDied","Data":"7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2"}
May 26 02:56:30.036840 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.036738    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc" event={"ID":"8fa41aea-c330-460d-8a3f-118ec9449f26","Type":"ContainerDied","Data":"0406e6f41420d83ec99fd99704e01d09cedbf95fc730234163041176957a37bd"}
May 26 02:56:30.036840 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.036758    2551 scope.go:117] "RemoveContainer" containerID="7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2"
May 26 02:56:30.036840 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.036707    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"
May 26 02:56:30.045010 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.044976    2551 scope.go:117] "RemoveContainer" containerID="7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2"
May 26 02:56:30.045267 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:56:30.045250    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2\": container with ID starting with 7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2 not found: ID does not exist" containerID="7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2"
May 26 02:56:30.045325 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.045274    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2"} err="failed to get container status \"7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2\": rpc error: code = NotFound desc = could not find container \"7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2\": container with ID starting with 7a860b335c694e89227c55d0f6ee45f2903d1fbfc392e769ff417d675f3072c2 not found: ID does not exist"
May 26 02:56:30.103759 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.103722    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-x2752\" (UniqueName: \"kubernetes.io/projected/8fa41aea-c330-460d-8a3f-118ec9449f26-kube-api-access-x2752\") pod \"8fa41aea-c330-460d-8a3f-118ec9449f26\" (UID: \"8fa41aea-c330-460d-8a3f-118ec9449f26\") "
May 26 02:56:30.106006 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.105979    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8fa41aea-c330-460d-8a3f-118ec9449f26-kube-api-access-x2752" (OuterVolumeSpecName: "kube-api-access-x2752") pod "8fa41aea-c330-460d-8a3f-118ec9449f26" (UID: "8fa41aea-c330-460d-8a3f-118ec9449f26"). InnerVolumeSpecName "kube-api-access-x2752". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:56:30.204765 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.204729    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-x2752\" (UniqueName: \"kubernetes.io/projected/8fa41aea-c330-460d-8a3f-118ec9449f26-kube-api-access-x2752\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:56:30.361686 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.361632    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"]
May 26 02:56:30.363386 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:30.363353    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["feast-operator-system/feast-operator-controller-manager-5f8946788d-tzhpc"]
May 26 02:56:31.043299 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:31.043214    2551 generic.go:358] "Generic (PLEG): container finished" podID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerID="574b525daf01edaf403dbb7869c1d4e90c30fc23d41c00776be001d815a7b4ad" exitCode=0
May 26 02:56:31.044514 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:31.044490    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8fa41aea-c330-460d-8a3f-118ec9449f26" path="/var/lib/kubelet/pods/8fa41aea-c330-460d-8a3f-118ec9449f26/volumes"
May 26 02:56:31.044794 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:31.044773    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" event={"ID":"bbd974f9-a60d-4707-963b-40715d92e3bb","Type":"ContainerDied","Data":"574b525daf01edaf403dbb7869c1d4e90c30fc23d41c00776be001d815a7b4ad"}
May 26 02:56:31.046241 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:31.046218    2551 generic.go:358] "Generic (PLEG): container finished" podID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerID="d8cee060888a2d1a0a81e81e4153effe4d3b067b53425ddde94168e461a77ef9" exitCode=0
May 26 02:56:31.046331 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:31.046246    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" event={"ID":"d3152c75-dc45-4de9-b0fb-880dcd9d9181","Type":"ContainerDied","Data":"d8cee060888a2d1a0a81e81e4153effe4d3b067b53425ddde94168e461a77ef9"}
May 26 02:56:31.728756 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:31.728719    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 26 02:56:32.423093 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.423056    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"]
May 26 02:56:32.423480 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.423249    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6" podUID="95d80b8d-f5fd-476b-b52b-c84fec865b68" containerName="postgres" containerID="cri-o://a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e" gracePeriod=30
May 26 02:56:32.427232 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.427205    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-g52h7"]
May 26 02:56:32.427452 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.427413    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/redis-854544c579-g52h7" podUID="eea90396-49c4-4c78-affc-b747df45631c" containerName="redis" containerID="cri-o://be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228" gracePeriod=30
May 26 02:56:32.687731 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.687710    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"
May 26 02:56:32.690768 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.690749    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-g52h7"
May 26 02:56:32.727441 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.727415    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-jcsnd\" (UniqueName: \"kubernetes.io/projected/95d80b8d-f5fd-476b-b52b-c84fec865b68-kube-api-access-jcsnd\") pod \"95d80b8d-f5fd-476b-b52b-c84fec865b68\" (UID: \"95d80b8d-f5fd-476b-b52b-c84fec865b68\") "
May 26 02:56:32.727602 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.727463    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-f7jfc\" (UniqueName: \"kubernetes.io/projected/eea90396-49c4-4c78-affc-b747df45631c-kube-api-access-f7jfc\") pod \"eea90396-49c4-4c78-affc-b747df45631c\" (UID: \"eea90396-49c4-4c78-affc-b747df45631c\") "
May 26 02:56:32.727602 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.727482    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/95d80b8d-f5fd-476b-b52b-c84fec865b68-postgresdata\") pod \"95d80b8d-f5fd-476b-b52b-c84fec865b68\" (UID: \"95d80b8d-f5fd-476b-b52b-c84fec865b68\") "
May 26 02:56:32.729700 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.729671    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/95d80b8d-f5fd-476b-b52b-c84fec865b68-kube-api-access-jcsnd" (OuterVolumeSpecName: "kube-api-access-jcsnd") pod "95d80b8d-f5fd-476b-b52b-c84fec865b68" (UID: "95d80b8d-f5fd-476b-b52b-c84fec865b68"). InnerVolumeSpecName "kube-api-access-jcsnd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:56:32.730785 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.730758    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/eea90396-49c4-4c78-affc-b747df45631c-kube-api-access-f7jfc" (OuterVolumeSpecName: "kube-api-access-f7jfc") pod "eea90396-49c4-4c78-affc-b747df45631c" (UID: "eea90396-49c4-4c78-affc-b747df45631c"). InnerVolumeSpecName "kube-api-access-f7jfc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:56:32.757385 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.757356    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/95d80b8d-f5fd-476b-b52b-c84fec865b68-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "95d80b8d-f5fd-476b-b52b-c84fec865b68" (UID: "95d80b8d-f5fd-476b-b52b-c84fec865b68"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 02:56:32.828362 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.828330    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-jcsnd\" (UniqueName: \"kubernetes.io/projected/95d80b8d-f5fd-476b-b52b-c84fec865b68-kube-api-access-jcsnd\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:56:32.828362 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.828358    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-f7jfc\" (UniqueName: \"kubernetes.io/projected/eea90396-49c4-4c78-affc-b747df45631c-kube-api-access-f7jfc\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:56:32.828362 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:32.828368    2551 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/95d80b8d-f5fd-476b-b52b-c84fec865b68-postgresdata\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:56:33.053655 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.053619    2551 generic.go:358] "Generic (PLEG): container finished" podID="eea90396-49c4-4c78-affc-b747df45631c" containerID="be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228" exitCode=0
May 26 02:56:33.053756 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.053657    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-g52h7" event={"ID":"eea90396-49c4-4c78-affc-b747df45631c","Type":"ContainerDied","Data":"be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228"}
May 26 02:56:33.053756 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.053675    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-g52h7"
May 26 02:56:33.053756 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.053692    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-g52h7" event={"ID":"eea90396-49c4-4c78-affc-b747df45631c","Type":"ContainerDied","Data":"b534c442c3225d4dbd2021ef207aeb7f579565e72c831776eb224eb615144136"}
May 26 02:56:33.053756 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.053711    2551 scope.go:117] "RemoveContainer" containerID="be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228"
May 26 02:56:33.055305 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.055177    2551 generic.go:358] "Generic (PLEG): container finished" podID="95d80b8d-f5fd-476b-b52b-c84fec865b68" containerID="a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e" exitCode=0
May 26 02:56:33.055305 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.055216    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6" event={"ID":"95d80b8d-f5fd-476b-b52b-c84fec865b68","Type":"ContainerDied","Data":"a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e"}
May 26 02:56:33.055305 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.055238    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6" event={"ID":"95d80b8d-f5fd-476b-b52b-c84fec865b68","Type":"ContainerDied","Data":"b7ec1c92eed6a4449160410ad6f7f529fe73220b43eaa02e2bf06ee89929ca0c"}
May 26 02:56:33.055305 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.055216    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"
May 26 02:56:33.062518 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.062502    2551 scope.go:117] "RemoveContainer" containerID="be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228"
May 26 02:56:33.062771 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:56:33.062750    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228\": container with ID starting with be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228 not found: ID does not exist" containerID="be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228"
May 26 02:56:33.062829 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.062778    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228"} err="failed to get container status \"be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228\": rpc error: code = NotFound desc = could not find container \"be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228\": container with ID starting with be887e1acf18744efe36ad8ede7b87bf598c999bf06f1cbdc9a8e46177dd6228 not found: ID does not exist"
May 26 02:56:33.062829 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.062793    2551 scope.go:117] "RemoveContainer" containerID="a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e"
May 26 02:56:33.070141 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.070123    2551 scope.go:117] "RemoveContainer" containerID="a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e"
May 26 02:56:33.070425 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:56:33.070407    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e\": container with ID starting with a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e not found: ID does not exist" containerID="a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e"
May 26 02:56:33.070487 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.070433    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e"} err="failed to get container status \"a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e\": rpc error: code = NotFound desc = could not find container \"a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e\": container with ID starting with a7a647e5789cd43209758181655aae8109214aa0cfcd5c126d1084e17c5af57e not found: ID does not exist"
May 26 02:56:33.071056 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.071038    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"]
May 26 02:56:33.075767 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.075746    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-2n2n6"]
May 26 02:56:33.084850 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.084831    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-g52h7"]
May 26 02:56:33.089016 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:33.088997    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-854544c579-g52h7"]
May 26 02:56:35.044225 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:35.044185    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="95d80b8d-f5fd-476b-b52b-c84fec865b68" path="/var/lib/kubelet/pods/95d80b8d-f5fd-476b-b52b-c84fec865b68/volumes"
May 26 02:56:35.044701 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:35.044647    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="eea90396-49c4-4c78-affc-b747df45631c" path="/var/lib/kubelet/pods/eea90396-49c4-4c78-affc-b747df45631c/volumes"
May 26 02:56:39.994519 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:39.994466    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 26 02:56:41.729381 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:41.729340    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 26 02:56:49.995014 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:49.994923    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 26 02:56:51.729464 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:51.729426    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 26 02:56:51.729866 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:51.729539    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:56:59.994798 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:59.994748    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 26 02:56:59.995223 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:56:59.994933    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:57:00.151621 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.151584    2551 generic.go:358] "Generic (PLEG): container finished" podID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerID="c2f23f9d1235846105d39bf6b7ab01cc25f6f6e566f754c54352bbdcd9ddc567" exitCode=137
May 26 02:57:00.151820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.151671    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" event={"ID":"d3152c75-dc45-4de9-b0fb-880dcd9d9181","Type":"ContainerDied","Data":"c2f23f9d1235846105d39bf6b7ab01cc25f6f6e566f754c54352bbdcd9ddc567"}
May 26 02:57:00.153699 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.153677    2551 generic.go:358] "Generic (PLEG): container finished" podID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerID="6a0dc69f7e92b8f76c817a25e959a47ccf01394b598cbb74276c6ecad604051e" exitCode=137
May 26 02:57:00.153809 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.153719    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" event={"ID":"bbd974f9-a60d-4707-963b-40715d92e3bb","Type":"ContainerDied","Data":"6a0dc69f7e92b8f76c817a25e959a47ccf01394b598cbb74276c6ecad604051e"}
May 26 02:57:00.599599 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.599573    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:57:00.633913 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.633872    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:57:00.775071 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.774985    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-registry-tls\") pod \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") "
May 26 02:57:00.775071 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.775053    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-pf6sw\" (UniqueName: \"kubernetes.io/projected/bbd974f9-a60d-4707-963b-40715d92e3bb-kube-api-access-pf6sw\") pod \"bbd974f9-a60d-4707-963b-40715d92e3bb\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") "
May 26 02:57:00.775071 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.775080    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-online-tls\") pod \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") "
May 26 02:57:00.775307 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.775127    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-qqwcx\" (UniqueName: \"kubernetes.io/projected/d3152c75-dc45-4de9-b0fb-880dcd9d9181-kube-api-access-qqwcx\") pod \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") "
May 26 02:57:00.775307 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.775143    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-online-tls\") pod \"bbd974f9-a60d-4707-963b-40715d92e3bb\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") "
May 26 02:57:00.775307 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.775191    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd974f9-a60d-4707-963b-40715d92e3bb-feast-data\") pod \"bbd974f9-a60d-4707-963b-40715d92e3bb\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") "
May 26 02:57:00.775307 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.775224    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d3152c75-dc45-4de9-b0fb-880dcd9d9181-feast-data\") pod \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\" (UID: \"d3152c75-dc45-4de9-b0fb-880dcd9d9181\") "
May 26 02:57:00.775307 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.775284    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-registry-tls\") pod \"bbd974f9-a60d-4707-963b-40715d92e3bb\" (UID: \"bbd974f9-a60d-4707-963b-40715d92e3bb\") "
May 26 02:57:00.775801 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.775722    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bbd974f9-a60d-4707-963b-40715d92e3bb-feast-data" (OuterVolumeSpecName: "feast-data") pod "bbd974f9-a60d-4707-963b-40715d92e3bb" (UID: "bbd974f9-a60d-4707-963b-40715d92e3bb"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 02:57:00.777751 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.777686    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-online-tls" (OuterVolumeSpecName: "online-tls") pod "d3152c75-dc45-4de9-b0fb-880dcd9d9181" (UID: "d3152c75-dc45-4de9-b0fb-880dcd9d9181"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:57:00.777751 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.777706    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bbd974f9-a60d-4707-963b-40715d92e3bb-kube-api-access-pf6sw" (OuterVolumeSpecName: "kube-api-access-pf6sw") pod "bbd974f9-a60d-4707-963b-40715d92e3bb" (UID: "bbd974f9-a60d-4707-963b-40715d92e3bb"). InnerVolumeSpecName "kube-api-access-pf6sw". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:57:00.777751 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.777718    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d3152c75-dc45-4de9-b0fb-880dcd9d9181" (UID: "d3152c75-dc45-4de9-b0fb-880dcd9d9181"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:57:00.777941 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.777779    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d3152c75-dc45-4de9-b0fb-880dcd9d9181-kube-api-access-qqwcx" (OuterVolumeSpecName: "kube-api-access-qqwcx") pod "d3152c75-dc45-4de9-b0fb-880dcd9d9181" (UID: "d3152c75-dc45-4de9-b0fb-880dcd9d9181"). InnerVolumeSpecName "kube-api-access-qqwcx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:57:00.777941 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.777782    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-online-tls" (OuterVolumeSpecName: "online-tls") pod "bbd974f9-a60d-4707-963b-40715d92e3bb" (UID: "bbd974f9-a60d-4707-963b-40715d92e3bb"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:57:00.777941 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.777806    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bbd974f9-a60d-4707-963b-40715d92e3bb" (UID: "bbd974f9-a60d-4707-963b-40715d92e3bb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:57:00.784446 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.784424    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d3152c75-dc45-4de9-b0fb-880dcd9d9181-feast-data" (OuterVolumeSpecName: "feast-data") pod "d3152c75-dc45-4de9-b0fb-880dcd9d9181" (UID: "d3152c75-dc45-4de9-b0fb-880dcd9d9181"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 02:57:00.876124 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.876099    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-qqwcx\" (UniqueName: \"kubernetes.io/projected/d3152c75-dc45-4de9-b0fb-880dcd9d9181-kube-api-access-qqwcx\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:57:00.876124 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.876124    2551 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-online-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:57:00.876271 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.876134    2551 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bbd974f9-a60d-4707-963b-40715d92e3bb-feast-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:57:00.876271 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.876142    2551 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d3152c75-dc45-4de9-b0fb-880dcd9d9181-feast-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:57:00.876271 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.876149    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/bbd974f9-a60d-4707-963b-40715d92e3bb-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:57:00.876271 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.876156    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:57:00.876271 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.876163    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-pf6sw\" (UniqueName: \"kubernetes.io/projected/bbd974f9-a60d-4707-963b-40715d92e3bb-kube-api-access-pf6sw\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:57:00.876271 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:00.876172    2551 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d3152c75-dc45-4de9-b0fb-880dcd9d9181-online-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:57:01.160013 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.159987    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"
May 26 02:57:01.160013 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.159992    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8" event={"ID":"bbd974f9-a60d-4707-963b-40715d92e3bb","Type":"ContainerDied","Data":"ecbe1c02f44b86ec999b5fd43e085731cb786de152dff9ac432534db6871f69b"}
May 26 02:57:01.160474 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.160039    2551 scope.go:117] "RemoveContainer" containerID="574b525daf01edaf403dbb7869c1d4e90c30fc23d41c00776be001d815a7b4ad"
May 26 02:57:01.162203 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.162172    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m" event={"ID":"d3152c75-dc45-4de9-b0fb-880dcd9d9181","Type":"ContainerDied","Data":"c3e930b01eff33749b3a33dc7beb60e4218e37e472b1b7262c78e41faa84084b"}
May 26 02:57:01.162431 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.162255    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"
May 26 02:57:01.168689 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.168672    2551 scope.go:117] "RemoveContainer" containerID="6a0dc69f7e92b8f76c817a25e959a47ccf01394b598cbb74276c6ecad604051e"
May 26 02:57:01.176746 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.176726    2551 scope.go:117] "RemoveContainer" containerID="eb9bea3d0678aab8558a49eb07c9bf82b70194e4a6fcfc3f528e1bad280a0b9a"
May 26 02:57:01.177164 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.177146    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"]
May 26 02:57:01.179814 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.179795    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-6cc7dbc64c-9j66m"]
May 26 02:57:01.184313 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.184297    2551 scope.go:117] "RemoveContainer" containerID="b50844a60d9a1d670db1194a06f653f68db7861691cf08ce6154ea2951d6e3fa"
May 26 02:57:01.190051 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.190033    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"]
May 26 02:57:01.192440 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.192421    2551 scope.go:117] "RemoveContainer" containerID="d8cee060888a2d1a0a81e81e4153effe4d3b067b53425ddde94168e461a77ef9"
May 26 02:57:01.193077 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.193059    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-7df4f69496-dclh8"]
May 26 02:57:01.199201 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.199184    2551 scope.go:117] "RemoveContainer" containerID="c2f23f9d1235846105d39bf6b7ab01cc25f6f6e566f754c54352bbdcd9ddc567"
May 26 02:57:01.206070 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.206052    2551 scope.go:117] "RemoveContainer" containerID="f87f97a00e7b9ecc76b76e7bc288f03c15ba8dc946e09a8e9ef74b764a97f308"
May 26 02:57:01.212399 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:01.212383    2551 scope.go:117] "RemoveContainer" containerID="ed0399c49d8cb111b0332325a8df136fc8171e981a6b4a448310bf8287eb9676"
May 26 02:57:03.044255 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:03.044227    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" path="/var/lib/kubelet/pods/bbd974f9-a60d-4707-963b-40715d92e3bb/volumes"
May 26 02:57:03.044888 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:03.044866    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" path="/var/lib/kubelet/pods/d3152c75-dc45-4de9-b0fb-880dcd9d9181/volumes"
May 26 02:57:08.044032 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.043996    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"]
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044292    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="feast-init"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044304    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="feast-init"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044316    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8fa41aea-c330-460d-8a3f-118ec9449f26" containerName="manager"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044321    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="8fa41aea-c330-460d-8a3f-118ec9449f26" containerName="manager"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044333    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="95d80b8d-f5fd-476b-b52b-c84fec865b68" containerName="postgres"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044343    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="95d80b8d-f5fd-476b-b52b-c84fec865b68" containerName="postgres"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044352    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="registry"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044358    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="registry"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044367    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="feast-apply"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044373    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="feast-apply"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044380    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="feast-apply"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044385    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="feast-apply"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044393    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="online"
May 26 02:57:08.044395 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044398    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="online"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044405    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="eea90396-49c4-4c78-affc-b747df45631c" containerName="redis"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044411    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="eea90396-49c4-4c78-affc-b747df45631c" containerName="redis"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044424    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="feast-init"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044431    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="feast-init"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044436    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="registry"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044440    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="registry"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044446    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="online"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044451    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="online"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044499    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="online"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044510    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="registry"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044516    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="bbd974f9-a60d-4707-963b-40715d92e3bb" containerName="online"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044523    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="eea90396-49c4-4c78-affc-b747df45631c" containerName="redis"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044529    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="d3152c75-dc45-4de9-b0fb-880dcd9d9181" containerName="registry"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044534    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="8fa41aea-c330-460d-8a3f-118ec9449f26" containerName="manager"
May 26 02:57:08.044820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.044542    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="95d80b8d-f5fd-476b-b52b-c84fec865b68" containerName="postgres"
May 26 02:57:08.050249 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.050231    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.053481 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.053456    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 26 02:57:08.053630 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.053466    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 26 02:57:08.053630 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.053508    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 26 02:57:08.053630 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.053521    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 26 02:57:08.053630 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.053558    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 26 02:57:08.053630 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.053575    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-fhz79\""
May 26 02:57:08.053917 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.053844    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 26 02:57:08.058987 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.058964    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"]
May 26 02:57:08.239195 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.239165    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfxn4\" (UniqueName: \"kubernetes.io/projected/c360eab4-0529-45ba-a622-dd0e9af10152-kube-api-access-dfxn4\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.239380 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.239211    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.239380 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.239252    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.239380 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.239279    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.239380 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.239324    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c360eab4-0529-45ba-a622-dd0e9af10152-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.239380 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.239368    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.340057 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.340026    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.340212 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.340083    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dfxn4\" (UniqueName: \"kubernetes.io/projected/c360eab4-0529-45ba-a622-dd0e9af10152-kube-api-access-dfxn4\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.340212 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.340116    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.340212 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.340138    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.340212 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.340156    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.340212 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.340175    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c360eab4-0529-45ba-a622-dd0e9af10152-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.340576 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.340550    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c360eab4-0529-45ba-a622-dd0e9af10152-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.342391 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.342370    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.342660 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.342634    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.342761 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.342634    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.342761 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.342689    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.347417 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.347399    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfxn4\" (UniqueName: \"kubernetes.io/projected/c360eab4-0529-45ba-a622-dd0e9af10152-kube-api-access-dfxn4\") pod \"feast-simple-feast-setup-664fcfcc58-255wj\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.369008 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.368989    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:08.492336 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:08.492311    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"]
May 26 02:57:08.494975 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:57:08.494934    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc360eab4_0529_45ba_a622_dd0e9af10152.slice/crio-41747b7f7e52385cb1cf9151a2ee5176768c8424b9d0aa672a37f2cb349cb35c WatchSource:0}: Error finding container 41747b7f7e52385cb1cf9151a2ee5176768c8424b9d0aa672a37f2cb349cb35c: Status 404 returned error can't find the container with id 41747b7f7e52385cb1cf9151a2ee5176768c8424b9d0aa672a37f2cb349cb35c
May 26 02:57:09.189189 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:09.189129    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerStarted","Data":"41747b7f7e52385cb1cf9151a2ee5176768c8424b9d0aa672a37f2cb349cb35c"}
May 26 02:57:22.241247 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:22.241152    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerStarted","Data":"fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8"}
May 26 02:57:27.259163 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:27.259129    2551 generic.go:358] "Generic (PLEG): container finished" podID="c360eab4-0529-45ba-a622-dd0e9af10152" containerID="fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8" exitCode=0
May 26 02:57:27.259556 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:27.259190    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerDied","Data":"fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8"}
May 26 02:57:28.265551 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:28.265513    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerStarted","Data":"9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238"}
May 26 02:57:33.284541 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:33.284506    2551 generic.go:358] "Generic (PLEG): container finished" podID="c360eab4-0529-45ba-a622-dd0e9af10152" containerID="9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238" exitCode=0
May 26 02:57:33.284973 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:33.284591    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerDied","Data":"9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238"}
May 26 02:57:34.292181 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:34.292137    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerStarted","Data":"d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361"}
May 26 02:57:34.292644 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:34.292188    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerStarted","Data":"0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb"}
May 26 02:57:34.292644 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:34.292205    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerStarted","Data":"07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548"}
May 26 02:57:34.292644 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:34.292218    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerStarted","Data":"4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2"}
May 26 02:57:34.316668 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:34.316581    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podStartSLOduration=12.867473536 podStartE2EDuration="26.316559359s" podCreationTimestamp="2026-05-26 02:57:08 +0000 UTC" firstStartedPulling="2026-05-26 02:57:08.496643348 +0000 UTC m=+1266.053431994" lastFinishedPulling="2026-05-26 02:57:21.94572917 +0000 UTC m=+1279.502517817" observedRunningTime="2026-05-26 02:57:34.313391686 +0000 UTC m=+1291.870180353" watchObservedRunningTime="2026-05-26 02:57:34.316559359 +0000 UTC m=+1291.873348034"
May 26 02:57:35.370065 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:35.369918    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:35.370065 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:35.369977    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:35.370065 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:35.369991    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:35.370065 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:35.370003    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:35.371775 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:35.371739    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.37:8816: connect: connection refused"
May 26 02:57:35.371941 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:35.371763    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.37:6571: connect: connection refused"
May 26 02:57:35.371941 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:35.371820    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 26 02:57:35.371941 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:35.371832    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 26 02:57:38.369134 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:38.369088    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:38.369640 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:38.369343    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.37:8816: connect: connection refused"
May 26 02:57:38.370068 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:38.370013    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.37:6571: connect: connection refused"
May 26 02:57:38.370068 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:38.370056    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:38.370260 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:38.370048    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 26 02:57:38.370260 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:38.370084    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:38.370260 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:38.370130    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:38.370514 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:38.370482    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 26 02:57:41.370456 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:41.370422    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:41.370922 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:41.370688    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:41.370922 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:41.370774    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:41.371367 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:41.371347    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:41.371464 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:41.371403    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:41.371533 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:41.371515    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:41.375018 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:41.375001    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:57:42.322650 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:57:42.322624    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:58:44.651759 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.651712    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"]
May 26 02:58:44.652344 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.652169    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="registry" containerID="cri-o://4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2" gracePeriod=30
May 26 02:58:44.652344 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.652214    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="ui" containerID="cri-o://d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361" gracePeriod=30
May 26 02:58:44.652344 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.652245    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="offline" containerID="cri-o://0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb" gracePeriod=30
May 26 02:58:44.652344 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.652231    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="online" containerID="cri-o://07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548" gracePeriod=30
May 26 02:58:44.920607 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.920519    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"]
May 26 02:58:44.923512 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.923492    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:44.925804 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.925783    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-wsm92\""
May 26 02:58:44.935751 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.935726    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"]
May 26 02:58:44.977661 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.977618    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/016f9316-d39d-4250-985a-e6d3ff008233-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:44.977867 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.977673    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:44.977867 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.977704    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-c6wrc\" (UniqueName: \"kubernetes.io/projected/016f9316-d39d-4250-985a-e6d3ff008233-kube-api-access-c6wrc\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:44.977867 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.977757    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:44.977867 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.977789    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:44.977867 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:44.977821    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.079151 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.079116    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/016f9316-d39d-4250-985a-e6d3ff008233-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.079359 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.079167    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.079359 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.079213    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-c6wrc\" (UniqueName: \"kubernetes.io/projected/016f9316-d39d-4250-985a-e6d3ff008233-kube-api-access-c6wrc\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.079359 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.079242    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.079359 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.079269    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.079544 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.079441    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.079591 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.079578    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/016f9316-d39d-4250-985a-e6d3ff008233-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.082462 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.082437    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.082619 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.082592    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.083169 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.083146    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.083653 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.083632    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.091031 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.091007    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-c6wrc\" (UniqueName: \"kubernetes.io/projected/016f9316-d39d-4250-985a-e6d3ff008233-kube-api-access-c6wrc\") pod \"feast-simple-feast-setup-664fcfcc58-fcb44\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.233308 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.233217    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:58:45.365309 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.365278    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"]
May 26 02:58:45.367728 ip-10-0-139-173 kubenswrapper[2551]: W0526 02:58:45.367696    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod016f9316_d39d_4250_985a_e6d3ff008233.slice/crio-17a006d750b465550715be22821e7f56697fa8ec6e4eb0818289dd32f26b7aad WatchSource:0}: Error finding container 17a006d750b465550715be22821e7f56697fa8ec6e4eb0818289dd32f26b7aad: Status 404 returned error can't find the container with id 17a006d750b465550715be22821e7f56697fa8ec6e4eb0818289dd32f26b7aad
May 26 02:58:45.517378 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.517288    2551 generic.go:358] "Generic (PLEG): container finished" podID="c360eab4-0529-45ba-a622-dd0e9af10152" containerID="d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361" exitCode=0
May 26 02:58:45.517378 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.517312    2551 generic.go:358] "Generic (PLEG): container finished" podID="c360eab4-0529-45ba-a622-dd0e9af10152" containerID="07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548" exitCode=0
May 26 02:58:45.517378 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.517352    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerDied","Data":"d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361"}
May 26 02:58:45.517684 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.517385    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerDied","Data":"07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548"}
May 26 02:58:45.518687 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.518661    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerStarted","Data":"f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773"}
May 26 02:58:45.518687 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:45.518686    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerStarted","Data":"17a006d750b465550715be22821e7f56697fa8ec6e4eb0818289dd32f26b7aad"}
May 26 02:58:50.537694 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:50.537658    2551 generic.go:358] "Generic (PLEG): container finished" podID="016f9316-d39d-4250-985a-e6d3ff008233" containerID="f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773" exitCode=0
May 26 02:58:50.538227 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:50.537698    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerDied","Data":"f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773"}
May 26 02:58:51.371689 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:51.371634    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 26 02:58:51.543361 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:51.543325    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerStarted","Data":"66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829"}
May 26 02:58:52.320527 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:52.320487    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 26 02:58:56.560499 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:56.560466    2551 generic.go:358] "Generic (PLEG): container finished" podID="016f9316-d39d-4250-985a-e6d3ff008233" containerID="66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829" exitCode=0
May 26 02:58:56.560882 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:56.560535    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerDied","Data":"66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829"}
May 26 02:58:57.567589 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:57.567539    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerStarted","Data":"cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3"}
May 26 02:58:57.568096 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:57.567617    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerStarted","Data":"0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940"}
May 26 02:58:57.568096 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:57.567636    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerStarted","Data":"6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7"}
May 26 02:58:57.568096 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:57.567667    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerStarted","Data":"6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6"}
May 26 02:58:57.592275 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:58:57.592217    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podStartSLOduration=13.592196927 podStartE2EDuration="13.592196927s" podCreationTimestamp="2026-05-26 02:58:44 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 02:58:57.590661672 +0000 UTC m=+1375.147450356" watchObservedRunningTime="2026-05-26 02:58:57.592196927 +0000 UTC m=+1375.148985598"
May 26 02:59:00.233360 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:00.233323    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:00.233936 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:00.233920    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:00.234077 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:00.234055    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:00.234077 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:00.234075    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:00.235735 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:00.235674    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.38:6571: connect: connection refused"
May 26 02:59:00.235855 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:00.235734    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 26 02:59:00.235855 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:00.235696    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.38:8816: connect: connection refused"
May 26 02:59:00.236087 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:00.236062    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 26 02:59:01.371974 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:01.371923    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 26 02:59:02.320195 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:02.320158    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 26 02:59:03.235012 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:03.234976    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:03.235461 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:03.235128    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:03.235461 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:03.235314    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:03.235645 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:03.235621    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:03.235731 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:03.235653    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:03.235930 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:03.235887    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:03.235988 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:03.235980    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:03.236054 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:03.236014    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:03.236054 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:03.236040    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:03.239707 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:03.239688    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:03.589552 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:03.589517    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:03.592442 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:03.592420    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 02:59:11.371544 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:11.371495    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 26 02:59:11.372001 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:11.371635    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:59:12.320591 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:12.320549    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 26 02:59:12.320762 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:12.320680    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:59:15.302948 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.302927    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:59:15.454174 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.454094    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-registry-tls\") pod \"c360eab4-0529-45ba-a622-dd0e9af10152\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") "
May 26 02:59:15.454174 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.454161    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-online-tls\") pod \"c360eab4-0529-45ba-a622-dd0e9af10152\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") "
May 26 02:59:15.454414 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.454181    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c360eab4-0529-45ba-a622-dd0e9af10152-feast-data\") pod \"c360eab4-0529-45ba-a622-dd0e9af10152\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") "
May 26 02:59:15.454414 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.454211    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-ui-tls\") pod \"c360eab4-0529-45ba-a622-dd0e9af10152\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") "
May 26 02:59:15.454414 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.454244    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-offline-tls\") pod \"c360eab4-0529-45ba-a622-dd0e9af10152\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") "
May 26 02:59:15.454414 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.454265    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfxn4\" (UniqueName: \"kubernetes.io/projected/c360eab4-0529-45ba-a622-dd0e9af10152-kube-api-access-dfxn4\") pod \"c360eab4-0529-45ba-a622-dd0e9af10152\" (UID: \"c360eab4-0529-45ba-a622-dd0e9af10152\") "
May 26 02:59:15.454786 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.454751    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c360eab4-0529-45ba-a622-dd0e9af10152-feast-data" (OuterVolumeSpecName: "feast-data") pod "c360eab4-0529-45ba-a622-dd0e9af10152" (UID: "c360eab4-0529-45ba-a622-dd0e9af10152"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 02:59:15.456509 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.456480    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c360eab4-0529-45ba-a622-dd0e9af10152" (UID: "c360eab4-0529-45ba-a622-dd0e9af10152"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:59:15.456619 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.456515    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-online-tls" (OuterVolumeSpecName: "online-tls") pod "c360eab4-0529-45ba-a622-dd0e9af10152" (UID: "c360eab4-0529-45ba-a622-dd0e9af10152"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:59:15.456619 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.456539    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c360eab4-0529-45ba-a622-dd0e9af10152" (UID: "c360eab4-0529-45ba-a622-dd0e9af10152"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:59:15.456716 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.456692    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c360eab4-0529-45ba-a622-dd0e9af10152" (UID: "c360eab4-0529-45ba-a622-dd0e9af10152"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 02:59:15.456820 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.456797    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c360eab4-0529-45ba-a622-dd0e9af10152-kube-api-access-dfxn4" (OuterVolumeSpecName: "kube-api-access-dfxn4") pod "c360eab4-0529-45ba-a622-dd0e9af10152" (UID: "c360eab4-0529-45ba-a622-dd0e9af10152"). InnerVolumeSpecName "kube-api-access-dfxn4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 02:59:15.555356 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.555329    2551 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-offline-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:59:15.555356 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.555352    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfxn4\" (UniqueName: \"kubernetes.io/projected/c360eab4-0529-45ba-a622-dd0e9af10152-kube-api-access-dfxn4\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:59:15.555507 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.555364    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:59:15.555507 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.555372    2551 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-online-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:59:15.555507 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.555380    2551 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c360eab4-0529-45ba-a622-dd0e9af10152-feast-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:59:15.555507 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.555389    2551 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c360eab4-0529-45ba-a622-dd0e9af10152-ui-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 02:59:15.630104 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.630068    2551 generic.go:358] "Generic (PLEG): container finished" podID="c360eab4-0529-45ba-a622-dd0e9af10152" containerID="0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb" exitCode=137
May 26 02:59:15.630104 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.630100    2551 generic.go:358] "Generic (PLEG): container finished" podID="c360eab4-0529-45ba-a622-dd0e9af10152" containerID="4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2" exitCode=137
May 26 02:59:15.630267 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.630136    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerDied","Data":"0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb"}
May 26 02:59:15.630267 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.630152    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"
May 26 02:59:15.630267 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.630173    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerDied","Data":"4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2"}
May 26 02:59:15.630267 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.630185    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj" event={"ID":"c360eab4-0529-45ba-a622-dd0e9af10152","Type":"ContainerDied","Data":"41747b7f7e52385cb1cf9151a2ee5176768c8424b9d0aa672a37f2cb349cb35c"}
May 26 02:59:15.630267 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.630200    2551 scope.go:117] "RemoveContainer" containerID="d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361"
May 26 02:59:15.638693 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.638676    2551 scope.go:117] "RemoveContainer" containerID="0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb"
May 26 02:59:15.646202 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.646183    2551 scope.go:117] "RemoveContainer" containerID="07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548"
May 26 02:59:15.650300 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.650278    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"]
May 26 02:59:15.654880 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.654854    2551 scope.go:117] "RemoveContainer" containerID="4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2"
May 26 02:59:15.655153 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.655053    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-255wj"]
May 26 02:59:15.661719 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.661557    2551 scope.go:117] "RemoveContainer" containerID="9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238"
May 26 02:59:15.668580 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.668565    2551 scope.go:117] "RemoveContainer" containerID="fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8"
May 26 02:59:15.675163 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.675147    2551 scope.go:117] "RemoveContainer" containerID="d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361"
May 26 02:59:15.675398 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:59:15.675381    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361\": container with ID starting with d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361 not found: ID does not exist" containerID="d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361"
May 26 02:59:15.675446 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.675407    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361"} err="failed to get container status \"d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361\": rpc error: code = NotFound desc = could not find container \"d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361\": container with ID starting with d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361 not found: ID does not exist"
May 26 02:59:15.675446 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.675425    2551 scope.go:117] "RemoveContainer" containerID="0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb"
May 26 02:59:15.675654 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:59:15.675635    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb\": container with ID starting with 0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb not found: ID does not exist" containerID="0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb"
May 26 02:59:15.675693 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.675660    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb"} err="failed to get container status \"0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb\": rpc error: code = NotFound desc = could not find container \"0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb\": container with ID starting with 0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb not found: ID does not exist"
May 26 02:59:15.675693 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.675678    2551 scope.go:117] "RemoveContainer" containerID="07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548"
May 26 02:59:15.675894 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:59:15.675877    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548\": container with ID starting with 07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548 not found: ID does not exist" containerID="07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548"
May 26 02:59:15.675961 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.675922    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548"} err="failed to get container status \"07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548\": rpc error: code = NotFound desc = could not find container \"07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548\": container with ID starting with 07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548 not found: ID does not exist"
May 26 02:59:15.675961 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.675937    2551 scope.go:117] "RemoveContainer" containerID="4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2"
May 26 02:59:15.676189 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:59:15.676172    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2\": container with ID starting with 4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2 not found: ID does not exist" containerID="4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2"
May 26 02:59:15.676241 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.676193    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2"} err="failed to get container status \"4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2\": rpc error: code = NotFound desc = could not find container \"4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2\": container with ID starting with 4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2 not found: ID does not exist"
May 26 02:59:15.676241 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.676207    2551 scope.go:117] "RemoveContainer" containerID="9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238"
May 26 02:59:15.676422 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:59:15.676401    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238\": container with ID starting with 9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238 not found: ID does not exist" containerID="9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238"
May 26 02:59:15.676467 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.676426    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238"} err="failed to get container status \"9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238\": rpc error: code = NotFound desc = could not find container \"9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238\": container with ID starting with 9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238 not found: ID does not exist"
May 26 02:59:15.676467 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.676440    2551 scope.go:117] "RemoveContainer" containerID="fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8"
May 26 02:59:15.676652 ip-10-0-139-173 kubenswrapper[2551]: E0526 02:59:15.676635    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8\": container with ID starting with fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8 not found: ID does not exist" containerID="fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8"
May 26 02:59:15.676701 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.676657    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8"} err="failed to get container status \"fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8\": rpc error: code = NotFound desc = could not find container \"fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8\": container with ID starting with fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8 not found: ID does not exist"
May 26 02:59:15.676701 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.676670    2551 scope.go:117] "RemoveContainer" containerID="d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361"
May 26 02:59:15.676883 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.676868    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361"} err="failed to get container status \"d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361\": rpc error: code = NotFound desc = could not find container \"d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361\": container with ID starting with d0cd94ce8ed088f3ed0a8d0a773df1c7c30740d7dc919518feae9a4f24fe8361 not found: ID does not exist"
May 26 02:59:15.676951 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.676884    2551 scope.go:117] "RemoveContainer" containerID="0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb"
May 26 02:59:15.677109 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.677093    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb"} err="failed to get container status \"0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb\": rpc error: code = NotFound desc = could not find container \"0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb\": container with ID starting with 0b2208f3770ade896808fc0db18f6cdf0e3ec1960434965acb2c5eba9cca7dcb not found: ID does not exist"
May 26 02:59:15.677179 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.677108    2551 scope.go:117] "RemoveContainer" containerID="07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548"
May 26 02:59:15.677313 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.677295    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548"} err="failed to get container status \"07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548\": rpc error: code = NotFound desc = could not find container \"07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548\": container with ID starting with 07cf66e3278fadf2d43fc7c6e65600f05f5a942235359491ed376a40eafa4548 not found: ID does not exist"
May 26 02:59:15.677356 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.677314    2551 scope.go:117] "RemoveContainer" containerID="4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2"
May 26 02:59:15.677507 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.677491    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2"} err="failed to get container status \"4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2\": rpc error: code = NotFound desc = could not find container \"4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2\": container with ID starting with 4c4d90ab358f4f516dd1e4a6969cce6890e3c44bf3258f1217fa002f232b3bc2 not found: ID does not exist"
May 26 02:59:15.677507 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.677506    2551 scope.go:117] "RemoveContainer" containerID="9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238"
May 26 02:59:15.677699 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.677682    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238"} err="failed to get container status \"9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238\": rpc error: code = NotFound desc = could not find container \"9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238\": container with ID starting with 9f89cdddc44ec9e27a52ed8dbf8de4122549cec96c57a78a26c463e3cdad9238 not found: ID does not exist"
May 26 02:59:15.677749 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.677700    2551 scope.go:117] "RemoveContainer" containerID="fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8"
May 26 02:59:15.677885 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:15.677866    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8"} err="failed to get container status \"fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8\": rpc error: code = NotFound desc = could not find container \"fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8\": container with ID starting with fcce3c90f70ce56185cb625f3b0e7a29cb390c3126e29b3e00af8280747d3bf8 not found: ID does not exist"
May 26 02:59:17.044263 ip-10-0-139-173 kubenswrapper[2551]: I0526 02:59:17.044230    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" path="/var/lib/kubelet/pods/c360eab4-0529-45ba-a622-dd0e9af10152/volumes"
May 26 03:00:06.737518 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737484    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"]
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737806    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="feast-init"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737820    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="feast-init"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737840    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="feast-apply"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737849    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="feast-apply"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737861    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="offline"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737869    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="offline"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737890    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="registry"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737908    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="registry"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737923    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="ui"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737931    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="ui"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737940    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="online"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.737945    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="online"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.738002    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="registry"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.738013    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="offline"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.738023    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="online"
May 26 03:00:06.738075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.738034    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="c360eab4-0529-45ba-a622-dd0e9af10152" containerName="ui"
May 26 03:00:06.742732 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.742712    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.745383 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.745208    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 26 03:00:06.745383 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.745225    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 26 03:00:06.745383 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.745226    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 26 03:00:06.745383 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.745281    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 26 03:00:06.745383 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.745228    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-rtx6c\""
May 26 03:00:06.745983 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.745960    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 26 03:00:06.745983 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.745977    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 26 03:00:06.752842 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.752822    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"]
May 26 03:00:06.883104 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.883064    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.883280 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.883118    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a3db5a83-0200-4d83-847b-0836500db684-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.883280 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.883155    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.883280 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.883204    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.883280 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.883230    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-gbfmj\" (UniqueName: \"kubernetes.io/projected/a3db5a83-0200-4d83-847b-0836500db684-kube-api-access-gbfmj\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.883456 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.883278    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a3db5a83-0200-4d83-847b-0836500db684-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.983648 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.983622    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.983832 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.983655    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a3db5a83-0200-4d83-847b-0836500db684-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.983832 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.983685    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.983832 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.983730    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.983832 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.983757    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-gbfmj\" (UniqueName: \"kubernetes.io/projected/a3db5a83-0200-4d83-847b-0836500db684-kube-api-access-gbfmj\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.983832 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:00:06.983777    2551 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 26 03:00:06.983832 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.983802    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a3db5a83-0200-4d83-847b-0836500db684-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.984200 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:00:06.983851    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-offline-tls podName:a3db5a83-0200-4d83-847b-0836500db684 nodeName:}" failed. No retries permitted until 2026-05-26 03:00:07.483830619 +0000 UTC m=+1445.040619287 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-offline-tls") pod "feast-simple-feast-remote-setup-5cff94746b-lf4kx" (UID: "a3db5a83-0200-4d83-847b-0836500db684") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 26 03:00:06.984200 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.984142    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a3db5a83-0200-4d83-847b-0836500db684-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.984546 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.984525    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a3db5a83-0200-4d83-847b-0836500db684-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.986291 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.986264    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.986401 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.986374    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:06.991687 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:06.991621    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-gbfmj\" (UniqueName: \"kubernetes.io/projected/a3db5a83-0200-4d83-847b-0836500db684-kube-api-access-gbfmj\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:07.488519 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:07.488467    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:07.490867 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:07.490846    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-lf4kx\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:07.653244 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:07.653207    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:07.781156 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:07.781126    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"]
May 26 03:00:07.782791 ip-10-0-139-173 kubenswrapper[2551]: W0526 03:00:07.782762    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda3db5a83_0200_4d83_847b_0836500db684.slice/crio-b8d2e9e6f7f3a6014cb997d1862b409b8ce53b1600f581e6b601d77a2de86a5b WatchSource:0}: Error finding container b8d2e9e6f7f3a6014cb997d1862b409b8ce53b1600f581e6b601d77a2de86a5b: Status 404 returned error can't find the container with id b8d2e9e6f7f3a6014cb997d1862b409b8ce53b1600f581e6b601d77a2de86a5b
May 26 03:00:07.784666 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:07.784649    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 26 03:00:08.788999 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:08.788951    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" event={"ID":"a3db5a83-0200-4d83-847b-0836500db684","Type":"ContainerStarted","Data":"14931c05e4c6abf01c27d58801aa6a68ff53a9ef4210efaa17148f69f162f36e"}
May 26 03:00:08.789354 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:08.789007    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" event={"ID":"a3db5a83-0200-4d83-847b-0836500db684","Type":"ContainerStarted","Data":"b8d2e9e6f7f3a6014cb997d1862b409b8ce53b1600f581e6b601d77a2de86a5b"}
May 26 03:00:12.802102 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:12.802068    2551 generic.go:358] "Generic (PLEG): container finished" podID="a3db5a83-0200-4d83-847b-0836500db684" containerID="14931c05e4c6abf01c27d58801aa6a68ff53a9ef4210efaa17148f69f162f36e" exitCode=0
May 26 03:00:12.802469 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:12.802141    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" event={"ID":"a3db5a83-0200-4d83-847b-0836500db684","Type":"ContainerDied","Data":"14931c05e4c6abf01c27d58801aa6a68ff53a9ef4210efaa17148f69f162f36e"}
May 26 03:00:13.806284 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:13.806252    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" event={"ID":"a3db5a83-0200-4d83-847b-0836500db684","Type":"ContainerStarted","Data":"0e77af3dff9313c476dd89e96ee3228226aa3696d9c53c50aa8f064e4f794319"}
May 26 03:00:18.822574 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:18.822545    2551 generic.go:358] "Generic (PLEG): container finished" podID="a3db5a83-0200-4d83-847b-0836500db684" containerID="0e77af3dff9313c476dd89e96ee3228226aa3696d9c53c50aa8f064e4f794319" exitCode=0
May 26 03:00:18.822962 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:18.822616    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" event={"ID":"a3db5a83-0200-4d83-847b-0836500db684","Type":"ContainerDied","Data":"0e77af3dff9313c476dd89e96ee3228226aa3696d9c53c50aa8f064e4f794319"}
May 26 03:00:19.828461 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:19.828424    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" event={"ID":"a3db5a83-0200-4d83-847b-0836500db684","Type":"ContainerStarted","Data":"7fab0a7d5bee62a81b3560f265a4949c157d20132406aeb5c8343672d82bee2d"}
May 26 03:00:19.828873 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:19.828466    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" event={"ID":"a3db5a83-0200-4d83-847b-0836500db684","Type":"ContainerStarted","Data":"464175ef26be951038b60155c6c97f5d2f322d14ecfa6ae46847abea33e4ebb5"}
May 26 03:00:19.828873 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:19.828482    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" event={"ID":"a3db5a83-0200-4d83-847b-0836500db684","Type":"ContainerStarted","Data":"eb45771a7a5cba9b9cbca4cc6640d81dba4d6622af05f4546f949fb41f641d3c"}
May 26 03:00:19.851833 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:19.851772    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podStartSLOduration=13.851751548 podStartE2EDuration="13.851751548s" podCreationTimestamp="2026-05-26 03:00:06 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 03:00:19.848930056 +0000 UTC m=+1457.405718727" watchObservedRunningTime="2026-05-26 03:00:19.851751548 +0000 UTC m=+1457.408540218"
May 26 03:00:22.654451 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:22.654390    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:22.654451 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:22.654441    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:22.654451 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:22.654458    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:22.656600 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:22.656539    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.39:8443: connect: connection refused"
May 26 03:00:22.656600 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:22.656557    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="online" probeResult="failure" output="Get \"https://10.134.0.39:6567/health\": dial tcp 10.134.0.39:6567: connect: connection refused"
May 26 03:00:22.656600 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:22.656585    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.39:8816: connect: connection refused"
May 26 03:00:25.655138 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:25.655106    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:25.655504 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:25.655274    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:25.655644 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:25.655625    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:25.655711 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:25.655655    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:25.655883 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:25.655865    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:25.656033 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:25.656019    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:25.659157 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:25.659141    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:25.850764 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:25.850727    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:00:25.853523 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:00:25.853503    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:01:03.016984 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:03.016956    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 03:01:03.019915 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:03.019873    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 03:01:03.021503 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:03.021487    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 03:01:03.024416 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:03.024399    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 03:01:27.944319 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:27.944280    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"]
May 26 03:01:27.944788 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:27.944705    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="online" containerID="cri-o://eb45771a7a5cba9b9cbca4cc6640d81dba4d6622af05f4546f949fb41f641d3c" gracePeriod=30
May 26 03:01:27.944858 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:27.944773    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="offline" containerID="cri-o://464175ef26be951038b60155c6c97f5d2f322d14ecfa6ae46847abea33e4ebb5" gracePeriod=30
May 26 03:01:27.944923 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:27.944774    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="ui" containerID="cri-o://7fab0a7d5bee62a81b3560f265a4949c157d20132406aeb5c8343672d82bee2d" gracePeriod=30
May 26 03:01:28.103804 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:28.103769    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"]
May 26 03:01:28.104283 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:28.104230    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="registry" containerID="cri-o://6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6" gracePeriod=30
May 26 03:01:28.104522 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:28.104292    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="ui" containerID="cri-o://cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3" gracePeriod=30
May 26 03:01:28.104643 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:28.104312    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="offline" containerID="cri-o://0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940" gracePeriod=30
May 26 03:01:28.104643 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:28.104308    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="online" containerID="cri-o://6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7" gracePeriod=30
May 26 03:01:29.038156 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:29.038073    2551 generic.go:358] "Generic (PLEG): container finished" podID="016f9316-d39d-4250-985a-e6d3ff008233" containerID="cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3" exitCode=0
May 26 03:01:29.038156 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:29.038099    2551 generic.go:358] "Generic (PLEG): container finished" podID="016f9316-d39d-4250-985a-e6d3ff008233" containerID="6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7" exitCode=0
May 26 03:01:29.038603 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:29.038151    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerDied","Data":"cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3"}
May 26 03:01:29.038603 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:29.038194    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerDied","Data":"6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7"}
May 26 03:01:29.040338 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:29.040316    2551 generic.go:358] "Generic (PLEG): container finished" podID="a3db5a83-0200-4d83-847b-0836500db684" containerID="7fab0a7d5bee62a81b3560f265a4949c157d20132406aeb5c8343672d82bee2d" exitCode=0
May 26 03:01:29.040338 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:29.040335    2551 generic.go:358] "Generic (PLEG): container finished" podID="a3db5a83-0200-4d83-847b-0836500db684" containerID="eb45771a7a5cba9b9cbca4cc6640d81dba4d6622af05f4546f949fb41f641d3c" exitCode=0
May 26 03:01:29.043719 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:29.043697    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" event={"ID":"a3db5a83-0200-4d83-847b-0836500db684","Type":"ContainerDied","Data":"7fab0a7d5bee62a81b3560f265a4949c157d20132406aeb5c8343672d82bee2d"}
May 26 03:01:29.043805 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:29.043722    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" event={"ID":"a3db5a83-0200-4d83-847b-0836500db684","Type":"ContainerDied","Data":"eb45771a7a5cba9b9cbca4cc6640d81dba4d6622af05f4546f949fb41f641d3c"}
May 26 03:01:33.236670 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:33.236627    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 26 03:01:33.590479 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:33.590438    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 26 03:01:35.656479 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:35.656434    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.39:8443: connect: connection refused"
May 26 03:01:35.851788 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:35.851751    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="online" probeResult="failure" output="Get \"https://10.134.0.39:6567/health\": dial tcp 10.134.0.39:6567: connect: connection refused"
May 26 03:01:43.235988 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:43.235944    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 26 03:01:43.590189 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:43.590149    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 26 03:01:45.656707 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:45.656661    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.39:8443: connect: connection refused"
May 26 03:01:45.851213 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:45.851172    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="online" probeResult="failure" output="Get \"https://10.134.0.39:6567/health\": dial tcp 10.134.0.39:6567: connect: connection refused"
May 26 03:01:53.236386 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:53.236337    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 26 03:01:53.238750 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:53.236478    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 03:01:53.589667 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:53.589633    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 26 03:01:53.589828 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:53.589768    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 03:01:55.656544 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:55.656498    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.39:8443: connect: connection refused"
May 26 03:01:55.656965 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:55.656639    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:01:55.851239 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:55.851196    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="online" probeResult="failure" output="Get \"https://10.134.0.39:6567/health\": dial tcp 10.134.0.39:6567: connect: connection refused"
May 26 03:01:55.851413 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:55.851304    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:01:58.121239 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.121212    2551 generic.go:358] "Generic (PLEG): container finished" podID="a3db5a83-0200-4d83-847b-0836500db684" containerID="464175ef26be951038b60155c6c97f5d2f322d14ecfa6ae46847abea33e4ebb5" exitCode=137
May 26 03:01:58.121626 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.121274    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" event={"ID":"a3db5a83-0200-4d83-847b-0836500db684","Type":"ContainerDied","Data":"464175ef26be951038b60155c6c97f5d2f322d14ecfa6ae46847abea33e4ebb5"}
May 26 03:01:58.594549 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.594525    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:01:58.740069 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.740036    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a3db5a83-0200-4d83-847b-0836500db684-registry-tls\") pod \"a3db5a83-0200-4d83-847b-0836500db684\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") "
May 26 03:01:58.740211 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.740074    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-offline-tls\") pod \"a3db5a83-0200-4d83-847b-0836500db684\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") "
May 26 03:01:58.740211 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.740100    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-gbfmj\" (UniqueName: \"kubernetes.io/projected/a3db5a83-0200-4d83-847b-0836500db684-kube-api-access-gbfmj\") pod \"a3db5a83-0200-4d83-847b-0836500db684\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") "
May 26 03:01:58.740211 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.740127    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-online-tls\") pod \"a3db5a83-0200-4d83-847b-0836500db684\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") "
May 26 03:01:58.740211 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.740145    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a3db5a83-0200-4d83-847b-0836500db684-feast-data\") pod \"a3db5a83-0200-4d83-847b-0836500db684\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") "
May 26 03:01:58.740211 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.740189    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-ui-tls\") pod \"a3db5a83-0200-4d83-847b-0836500db684\" (UID: \"a3db5a83-0200-4d83-847b-0836500db684\") "
May 26 03:01:58.740489 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.740459    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/a3db5a83-0200-4d83-847b-0836500db684-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a3db5a83-0200-4d83-847b-0836500db684" (UID: "a3db5a83-0200-4d83-847b-0836500db684"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 03:01:58.740783 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.740752    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a3db5a83-0200-4d83-847b-0836500db684-feast-data" (OuterVolumeSpecName: "feast-data") pod "a3db5a83-0200-4d83-847b-0836500db684" (UID: "a3db5a83-0200-4d83-847b-0836500db684"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 03:01:58.742476 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.742449    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-online-tls" (OuterVolumeSpecName: "online-tls") pod "a3db5a83-0200-4d83-847b-0836500db684" (UID: "a3db5a83-0200-4d83-847b-0836500db684"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:01:58.742583 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.742506    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a3db5a83-0200-4d83-847b-0836500db684-kube-api-access-gbfmj" (OuterVolumeSpecName: "kube-api-access-gbfmj") pod "a3db5a83-0200-4d83-847b-0836500db684" (UID: "a3db5a83-0200-4d83-847b-0836500db684"). InnerVolumeSpecName "kube-api-access-gbfmj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 03:01:58.742583 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.742533    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a3db5a83-0200-4d83-847b-0836500db684" (UID: "a3db5a83-0200-4d83-847b-0836500db684"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:01:58.742583 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.742545    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "a3db5a83-0200-4d83-847b-0836500db684" (UID: "a3db5a83-0200-4d83-847b-0836500db684"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:01:58.746620 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.746603    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 03:01:58.841132 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841097    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-offline-tls\") pod \"016f9316-d39d-4250-985a-e6d3ff008233\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") "
May 26 03:01:58.841302 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841149    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-online-tls\") pod \"016f9316-d39d-4250-985a-e6d3ff008233\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") "
May 26 03:01:58.841302 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841169    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-registry-tls\") pod \"016f9316-d39d-4250-985a-e6d3ff008233\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") "
May 26 03:01:58.841302 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841228    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/016f9316-d39d-4250-985a-e6d3ff008233-feast-data\") pod \"016f9316-d39d-4250-985a-e6d3ff008233\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") "
May 26 03:01:58.841302 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841266    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-c6wrc\" (UniqueName: \"kubernetes.io/projected/016f9316-d39d-4250-985a-e6d3ff008233-kube-api-access-c6wrc\") pod \"016f9316-d39d-4250-985a-e6d3ff008233\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") "
May 26 03:01:58.841482 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841303    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-ui-tls\") pod \"016f9316-d39d-4250-985a-e6d3ff008233\" (UID: \"016f9316-d39d-4250-985a-e6d3ff008233\") "
May 26 03:01:58.841602 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841576    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/a3db5a83-0200-4d83-847b-0836500db684-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:01:58.841602 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841601    2551 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-offline-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:01:58.841802 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841614    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-gbfmj\" (UniqueName: \"kubernetes.io/projected/a3db5a83-0200-4d83-847b-0836500db684-kube-api-access-gbfmj\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:01:58.841802 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841630    2551 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-online-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:01:58.841802 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841644    2551 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a3db5a83-0200-4d83-847b-0836500db684-feast-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:01:58.841802 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841658    2551 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a3db5a83-0200-4d83-847b-0836500db684-ui-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:01:58.842041 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.841818    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/016f9316-d39d-4250-985a-e6d3ff008233-feast-data" (OuterVolumeSpecName: "feast-data") pod "016f9316-d39d-4250-985a-e6d3ff008233" (UID: "016f9316-d39d-4250-985a-e6d3ff008233"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 03:01:58.843511 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.843482    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "016f9316-d39d-4250-985a-e6d3ff008233" (UID: "016f9316-d39d-4250-985a-e6d3ff008233"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:01:58.843624 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.843508    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-online-tls" (OuterVolumeSpecName: "online-tls") pod "016f9316-d39d-4250-985a-e6d3ff008233" (UID: "016f9316-d39d-4250-985a-e6d3ff008233"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:01:58.843624 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.843564    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/016f9316-d39d-4250-985a-e6d3ff008233-kube-api-access-c6wrc" (OuterVolumeSpecName: "kube-api-access-c6wrc") pod "016f9316-d39d-4250-985a-e6d3ff008233" (UID: "016f9316-d39d-4250-985a-e6d3ff008233"). InnerVolumeSpecName "kube-api-access-c6wrc". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 03:01:58.843803 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.843784    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "016f9316-d39d-4250-985a-e6d3ff008233" (UID: "016f9316-d39d-4250-985a-e6d3ff008233"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:01:58.843881 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.843805    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "016f9316-d39d-4250-985a-e6d3ff008233" (UID: "016f9316-d39d-4250-985a-e6d3ff008233"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:01:58.942318 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.942286    2551 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-online-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:01:58.942318 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.942314    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:01:58.942318 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.942325    2551 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/016f9316-d39d-4250-985a-e6d3ff008233-feast-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:01:58.942521 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.942334    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-c6wrc\" (UniqueName: \"kubernetes.io/projected/016f9316-d39d-4250-985a-e6d3ff008233-kube-api-access-c6wrc\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:01:58.942521 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.942344    2551 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-ui-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:01:58.942521 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:58.942352    2551 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/016f9316-d39d-4250-985a-e6d3ff008233-offline-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:01:59.126772 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.126747    2551 generic.go:358] "Generic (PLEG): container finished" podID="016f9316-d39d-4250-985a-e6d3ff008233" containerID="0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940" exitCode=137
May 26 03:01:59.126772 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.126769    2551 generic.go:358] "Generic (PLEG): container finished" podID="016f9316-d39d-4250-985a-e6d3ff008233" containerID="6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6" exitCode=137
May 26 03:01:59.127231 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.126822    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"
May 26 03:01:59.127231 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.126820    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerDied","Data":"0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940"}
May 26 03:01:59.127231 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.126864    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerDied","Data":"6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6"}
May 26 03:01:59.127231 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.126879    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44" event={"ID":"016f9316-d39d-4250-985a-e6d3ff008233","Type":"ContainerDied","Data":"17a006d750b465550715be22821e7f56697fa8ec6e4eb0818289dd32f26b7aad"}
May 26 03:01:59.127231 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.126914    2551 scope.go:117] "RemoveContainer" containerID="cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3"
May 26 03:01:59.129443 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.129421    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx" event={"ID":"a3db5a83-0200-4d83-847b-0836500db684","Type":"ContainerDied","Data":"b8d2e9e6f7f3a6014cb997d1862b409b8ce53b1600f581e6b601d77a2de86a5b"}
May 26 03:01:59.129547 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.129493    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"
May 26 03:01:59.136099 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.136082    2551 scope.go:117] "RemoveContainer" containerID="0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940"
May 26 03:01:59.143377 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.143359    2551 scope.go:117] "RemoveContainer" containerID="6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7"
May 26 03:01:59.144742 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.144723    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"]
May 26 03:01:59.148154 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.148134    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-fcb44"]
May 26 03:01:59.150733 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.150713    2551 scope.go:117] "RemoveContainer" containerID="6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6"
May 26 03:01:59.157267 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.157250    2551 scope.go:117] "RemoveContainer" containerID="66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829"
May 26 03:01:59.160141 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.160120    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"]
May 26 03:01:59.163497 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.163468    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-lf4kx"]
May 26 03:01:59.164673 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.164657    2551 scope.go:117] "RemoveContainer" containerID="f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773"
May 26 03:01:59.170989 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.170970    2551 scope.go:117] "RemoveContainer" containerID="cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3"
May 26 03:01:59.171212 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:01:59.171196    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3\": container with ID starting with cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3 not found: ID does not exist" containerID="cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3"
May 26 03:01:59.171268 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.171218    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3"} err="failed to get container status \"cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3\": rpc error: code = NotFound desc = could not find container \"cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3\": container with ID starting with cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3 not found: ID does not exist"
May 26 03:01:59.171268 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.171234    2551 scope.go:117] "RemoveContainer" containerID="0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940"
May 26 03:01:59.171432 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:01:59.171415    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940\": container with ID starting with 0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940 not found: ID does not exist" containerID="0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940"
May 26 03:01:59.171495 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.171436    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940"} err="failed to get container status \"0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940\": rpc error: code = NotFound desc = could not find container \"0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940\": container with ID starting with 0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940 not found: ID does not exist"
May 26 03:01:59.171495 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.171449    2551 scope.go:117] "RemoveContainer" containerID="6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7"
May 26 03:01:59.171626 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:01:59.171611    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7\": container with ID starting with 6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7 not found: ID does not exist" containerID="6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7"
May 26 03:01:59.171682 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.171629    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7"} err="failed to get container status \"6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7\": rpc error: code = NotFound desc = could not find container \"6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7\": container with ID starting with 6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7 not found: ID does not exist"
May 26 03:01:59.171682 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.171641    2551 scope.go:117] "RemoveContainer" containerID="6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6"
May 26 03:01:59.171846 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:01:59.171829    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6\": container with ID starting with 6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6 not found: ID does not exist" containerID="6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6"
May 26 03:01:59.171884 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.171853    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6"} err="failed to get container status \"6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6\": rpc error: code = NotFound desc = could not find container \"6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6\": container with ID starting with 6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6 not found: ID does not exist"
May 26 03:01:59.171884 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.171865    2551 scope.go:117] "RemoveContainer" containerID="66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829"
May 26 03:01:59.172089 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:01:59.172070    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829\": container with ID starting with 66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829 not found: ID does not exist" containerID="66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829"
May 26 03:01:59.172146 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.172104    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829"} err="failed to get container status \"66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829\": rpc error: code = NotFound desc = could not find container \"66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829\": container with ID starting with 66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829 not found: ID does not exist"
May 26 03:01:59.172146 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.172121    2551 scope.go:117] "RemoveContainer" containerID="f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773"
May 26 03:01:59.172361 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:01:59.172340    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773\": container with ID starting with f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773 not found: ID does not exist" containerID="f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773"
May 26 03:01:59.172441 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.172365    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773"} err="failed to get container status \"f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773\": rpc error: code = NotFound desc = could not find container \"f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773\": container with ID starting with f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773 not found: ID does not exist"
May 26 03:01:59.172441 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.172379    2551 scope.go:117] "RemoveContainer" containerID="cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3"
May 26 03:01:59.172572 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.172555    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3"} err="failed to get container status \"cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3\": rpc error: code = NotFound desc = could not find container \"cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3\": container with ID starting with cfa03474bffc279cf99dd42dde0a2e7270042e6548325e37c5f0654fb9225da3 not found: ID does not exist"
May 26 03:01:59.172631 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.172572    2551 scope.go:117] "RemoveContainer" containerID="0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940"
May 26 03:01:59.172769 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.172749    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940"} err="failed to get container status \"0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940\": rpc error: code = NotFound desc = could not find container \"0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940\": container with ID starting with 0af20eb366ed45fd2c776e7956180f58a4b9e1b64ded69d45608c119a8c82940 not found: ID does not exist"
May 26 03:01:59.172769 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.172768    2551 scope.go:117] "RemoveContainer" containerID="6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7"
May 26 03:01:59.172987 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.172966    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7"} err="failed to get container status \"6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7\": rpc error: code = NotFound desc = could not find container \"6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7\": container with ID starting with 6be0dc8856e0fa1c62cd5748f49c73841841c449e9d45b10f3e2f0379f8d1ad7 not found: ID does not exist"
May 26 03:01:59.173075 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.172990    2551 scope.go:117] "RemoveContainer" containerID="6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6"
May 26 03:01:59.173224 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.173205    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6"} err="failed to get container status \"6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6\": rpc error: code = NotFound desc = could not find container \"6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6\": container with ID starting with 6b970c1ea6e59a03fb653778f0b036f99de4a9700c8a3b3ff26f66d6419c74b6 not found: ID does not exist"
May 26 03:01:59.173275 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.173226    2551 scope.go:117] "RemoveContainer" containerID="66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829"
May 26 03:01:59.173457 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.173439    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829"} err="failed to get container status \"66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829\": rpc error: code = NotFound desc = could not find container \"66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829\": container with ID starting with 66a81ce1acd2687a183e598f236eee7d8b55cced328b2dfdacaf86161f5f8829 not found: ID does not exist"
May 26 03:01:59.173537 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.173458    2551 scope.go:117] "RemoveContainer" containerID="f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773"
May 26 03:01:59.173677 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.173660    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773"} err="failed to get container status \"f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773\": rpc error: code = NotFound desc = could not find container \"f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773\": container with ID starting with f06056f537a142361d32aefe8463fedead77b9eed69642253b62e9a0e53e9773 not found: ID does not exist"
May 26 03:01:59.173729 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.173678    2551 scope.go:117] "RemoveContainer" containerID="7fab0a7d5bee62a81b3560f265a4949c157d20132406aeb5c8343672d82bee2d"
May 26 03:01:59.180367 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.180353    2551 scope.go:117] "RemoveContainer" containerID="464175ef26be951038b60155c6c97f5d2f322d14ecfa6ae46847abea33e4ebb5"
May 26 03:01:59.186832 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.186810    2551 scope.go:117] "RemoveContainer" containerID="eb45771a7a5cba9b9cbca4cc6640d81dba4d6622af05f4546f949fb41f641d3c"
May 26 03:01:59.193080 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.193066    2551 scope.go:117] "RemoveContainer" containerID="0e77af3dff9313c476dd89e96ee3228226aa3696d9c53c50aa8f064e4f794319"
May 26 03:01:59.199385 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:01:59.199369    2551 scope.go:117] "RemoveContainer" containerID="14931c05e4c6abf01c27d58801aa6a68ff53a9ef4210efaa17148f69f162f36e"
May 26 03:02:01.045610 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:01.045578    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="016f9316-d39d-4250-985a-e6d3ff008233" path="/var/lib/kubelet/pods/016f9316-d39d-4250-985a-e6d3ff008233/volumes"
May 26 03:02:01.046220 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:01.046204    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a3db5a83-0200-4d83-847b-0836500db684" path="/var/lib/kubelet/pods/a3db5a83-0200-4d83-847b-0836500db684/volumes"
May 26 03:02:37.356994 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.356959    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"]
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357405    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="registry"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357419    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="registry"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357440    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="feast-apply"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357449    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="feast-apply"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357459    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="ui"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357467    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="ui"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357479    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="feast-init"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357487    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="feast-init"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357504    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="feast-init"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357512    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="feast-init"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357522    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="online"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357530    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="online"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357538    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="offline"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357547    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="offline"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357559    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="online"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357567    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="online"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357578    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="ui"
May 26 03:02:37.357576 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357585    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="ui"
May 26 03:02:37.358404 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357597    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="feast-apply"
May 26 03:02:37.358404 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357606    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="feast-apply"
May 26 03:02:37.358404 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357620    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="offline"
May 26 03:02:37.358404 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357628    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="offline"
May 26 03:02:37.358404 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357713    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="online"
May 26 03:02:37.358404 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357724    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="offline"
May 26 03:02:37.358404 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357732    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="ui"
May 26 03:02:37.358404 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357742    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="ui"
May 26 03:02:37.358404 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357753    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="016f9316-d39d-4250-985a-e6d3ff008233" containerName="registry"
May 26 03:02:37.358404 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357762    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="offline"
May 26 03:02:37.358404 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.357772    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="a3db5a83-0200-4d83-847b-0836500db684" containerName="online"
May 26 03:02:37.361677 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.361659    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.363850 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.363817    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 26 03:02:37.364050 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.364030    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 26 03:02:37.364132 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.364057    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 26 03:02:37.364208 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.364193    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 26 03:02:37.364264 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.364210    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 26 03:02:37.364596 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.364580    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 26 03:02:37.364655 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.364584    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-6xrfw\""
May 26 03:02:37.370857 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.370837    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"]
May 26 03:02:37.458759 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.458723    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab722eb4-e466-4691-9b30-27275d57b2c3-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.458982 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.458781    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.458982 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.458844    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.458982 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.458890    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.459141 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.458982    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-czmdn\" (UniqueName: \"kubernetes.io/projected/ab722eb4-e466-4691-9b30-27275d57b2c3-kube-api-access-czmdn\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.459141 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.459017    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.560116 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.560084    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-czmdn\" (UniqueName: \"kubernetes.io/projected/ab722eb4-e466-4691-9b30-27275d57b2c3-kube-api-access-czmdn\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.560286 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.560125    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.560286 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.560183    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab722eb4-e466-4691-9b30-27275d57b2c3-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.560286 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.560224    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.560394 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.560276    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.560394 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.560328    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.560463 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:02:37.560451    2551 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 26 03:02:37.560534 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:02:37.560524    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-ui-tls podName:ab722eb4-e466-4691-9b30-27275d57b2c3 nodeName:}" failed. No retries permitted until 2026-05-26 03:02:38.060501259 +0000 UTC m=+1595.617289910 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-fd4bg" (UID: "ab722eb4-e466-4691-9b30-27275d57b2c3") : secret "feast-simple-feast-setup-ui-tls" not found
May 26 03:02:37.560600 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.560559    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab722eb4-e466-4691-9b30-27275d57b2c3-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.562916 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.562879    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.563034 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.562986    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.563034 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.562996    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:37.567551 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:37.567526    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-czmdn\" (UniqueName: \"kubernetes.io/projected/ab722eb4-e466-4691-9b30-27275d57b2c3-kube-api-access-czmdn\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:38.064429 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:38.064392    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:38.067227 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:38.067203    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-fd4bg\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:38.280349 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:38.280318    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:38.410355 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:38.410322    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"]
May 26 03:02:38.410882 ip-10-0-139-173 kubenswrapper[2551]: W0526 03:02:38.410856    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podab722eb4_e466_4691_9b30_27275d57b2c3.slice/crio-7cdfecb391119940b56ef8388695808470a4354243567325dbf91fe8c475599c WatchSource:0}: Error finding container 7cdfecb391119940b56ef8388695808470a4354243567325dbf91fe8c475599c: Status 404 returned error can't find the container with id 7cdfecb391119940b56ef8388695808470a4354243567325dbf91fe8c475599c
May 26 03:02:39.246230 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:39.246191    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerStarted","Data":"66ab5182af638c8ce3e223b072e84229b8926bd4d41c29408976c24a6aba5fb2"}
May 26 03:02:39.246230 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:39.246235    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerStarted","Data":"7cdfecb391119940b56ef8388695808470a4354243567325dbf91fe8c475599c"}
May 26 03:02:43.260995 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:43.260914    2551 generic.go:358] "Generic (PLEG): container finished" podID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerID="66ab5182af638c8ce3e223b072e84229b8926bd4d41c29408976c24a6aba5fb2" exitCode=0
May 26 03:02:43.260995 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:43.260980    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerDied","Data":"66ab5182af638c8ce3e223b072e84229b8926bd4d41c29408976c24a6aba5fb2"}
May 26 03:02:44.265819 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:44.265784    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerStarted","Data":"8243d027bb7c64c59ef23b03bef5907e00da9c21f813fc8a789c2044807885f5"}
May 26 03:02:49.280584 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:49.280547    2551 generic.go:358] "Generic (PLEG): container finished" podID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerID="8243d027bb7c64c59ef23b03bef5907e00da9c21f813fc8a789c2044807885f5" exitCode=0
May 26 03:02:49.280982 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:49.280628    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerDied","Data":"8243d027bb7c64c59ef23b03bef5907e00da9c21f813fc8a789c2044807885f5"}
May 26 03:02:50.288625 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:50.288579    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerStarted","Data":"4fdbec5c96e1b68137b440567c57490b51f4c1c8f0fb005161286a40f39e1f60"}
May 26 03:02:50.288625 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:50.288629    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerStarted","Data":"f99470d4b0113e15614dc88625d4eaa2e61cef38ba84aabea1c5ff26e2faf9b3"}
May 26 03:02:50.289216 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:50.288643    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerStarted","Data":"9cdbb95818d33d2d96d0d52e1855181833855e4b25e64881f9f93d178d37a50b"}
May 26 03:02:50.289216 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:50.288656    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerStarted","Data":"4787928d8d027f195192550559d62afd14519a9d4f26741804035a45545a27b5"}
May 26 03:02:50.310555 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:50.310496    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podStartSLOduration=13.310477384 podStartE2EDuration="13.310477384s" podCreationTimestamp="2026-05-26 03:02:37 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 03:02:50.308746063 +0000 UTC m=+1607.865534735" watchObservedRunningTime="2026-05-26 03:02:50.310477384 +0000 UTC m=+1607.867266057"
May 26 03:02:53.280587 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:53.280549    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:53.280587 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:53.280594    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:53.281145 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:53.280609    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:53.281622 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:53.281592    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:53.282786 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:53.282751    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.40:8816: connect: connection refused"
May 26 03:02:53.282920 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:53.282828    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.40:6567/health\": dial tcp 10.134.0.40:6567: connect: connection refused"
May 26 03:02:53.282920 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:53.282830    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.40:6571: connect: connection refused"
May 26 03:02:53.283112 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:53.283072    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.40:8443: connect: connection refused"
May 26 03:02:56.281833 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:56.281799    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:56.282316 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:56.282082    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:56.282602 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:56.282575    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:56.282713 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:56.282611    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:56.282713 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:56.282622    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:56.282713 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:56.282659    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:56.282991 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:56.282975    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:56.286605 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:56.286585    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:56.310951 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:56.310927    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:56.311128 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:56.311110    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:56.313772 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:56.313751    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:02:57.314728 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:02:57.314696    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:03:58.297064 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.297027    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"]
May 26 03:03:58.297647 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.297472    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="registry" containerID="cri-o://4787928d8d027f195192550559d62afd14519a9d4f26741804035a45545a27b5" gracePeriod=30
May 26 03:03:58.297647 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.297592    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="offline" containerID="cri-o://f99470d4b0113e15614dc88625d4eaa2e61cef38ba84aabea1c5ff26e2faf9b3" gracePeriod=30
May 26 03:03:58.297647 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.297606    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="online" containerID="cri-o://9cdbb95818d33d2d96d0d52e1855181833855e4b25e64881f9f93d178d37a50b" gracePeriod=30
May 26 03:03:58.297807 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.297689    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="ui" containerID="cri-o://4fdbec5c96e1b68137b440567c57490b51f4c1c8f0fb005161286a40f39e1f60" gracePeriod=30
May 26 03:03:58.536598 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.536556    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"]
May 26 03:03:58.541054 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.541029    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.543188 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.543162    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-bj52p\""
May 26 03:03:58.554563 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.554533    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"]
May 26 03:03:58.655561 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.655476    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.655761 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.655604    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.655761 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.655692    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.655883 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.655775    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5aec-e227-435b-9629-f2d716c329f3-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.655883 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.655843    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.656020 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.655874    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-t2hpp\" (UniqueName: \"kubernetes.io/projected/9ccf5aec-e227-435b-9629-f2d716c329f3-kube-api-access-t2hpp\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.756772 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.756718    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.756971 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.756835    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5aec-e227-435b-9629-f2d716c329f3-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.756971 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.756943    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.757108 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.756990    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-t2hpp\" (UniqueName: \"kubernetes.io/projected/9ccf5aec-e227-435b-9629-f2d716c329f3-kube-api-access-t2hpp\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.757108 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.757040    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.757108 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.757065    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.757266 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:03:58.757154    2551 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 26 03:03:58.757266 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:03:58.757159    2551 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 26 03:03:58.757266 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:03:58.757235    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-registry-tls podName:9ccf5aec-e227-435b-9629-f2d716c329f3 nodeName:}" failed. No retries permitted until 2026-05-26 03:03:59.257212977 +0000 UTC m=+1676.814001629 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-registry-tls") pod "feast-simple-feast-setup-664fcfcc58-qlntc" (UID: "9ccf5aec-e227-435b-9629-f2d716c329f3") : secret "feast-simple-feast-setup-registry-tls" not found
May 26 03:03:58.757266 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:03:58.757257    2551 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-online-tls podName:9ccf5aec-e227-435b-9629-f2d716c329f3 nodeName:}" failed. No retries permitted until 2026-05-26 03:03:59.257245709 +0000 UTC m=+1676.814034356 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-online-tls") pod "feast-simple-feast-setup-664fcfcc58-qlntc" (UID: "9ccf5aec-e227-435b-9629-f2d716c329f3") : secret "feast-simple-feast-setup-online-tls" not found
May 26 03:03:58.757642 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.757616    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5aec-e227-435b-9629-f2d716c329f3-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.759629 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.759601    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.759629 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.759618    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:58.765367 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:58.765345    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-t2hpp\" (UniqueName: \"kubernetes.io/projected/9ccf5aec-e227-435b-9629-f2d716c329f3-kube-api-access-t2hpp\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:59.262971 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:59.262865    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:59.262971 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:59.262959    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:59.265634 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:59.265598    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:59.266015 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:59.265988    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-qlntc\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:59.454293 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:59.454256    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:03:59.496298 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:59.496262    2551 generic.go:358] "Generic (PLEG): container finished" podID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerID="4fdbec5c96e1b68137b440567c57490b51f4c1c8f0fb005161286a40f39e1f60" exitCode=0
May 26 03:03:59.496298 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:59.496288    2551 generic.go:358] "Generic (PLEG): container finished" podID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerID="9cdbb95818d33d2d96d0d52e1855181833855e4b25e64881f9f93d178d37a50b" exitCode=0
May 26 03:03:59.496547 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:59.496317    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerDied","Data":"4fdbec5c96e1b68137b440567c57490b51f4c1c8f0fb005161286a40f39e1f60"}
May 26 03:03:59.496547 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:59.496340    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerDied","Data":"9cdbb95818d33d2d96d0d52e1855181833855e4b25e64881f9f93d178d37a50b"}
May 26 03:03:59.581303 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:03:59.581277    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"]
May 26 03:03:59.583281 ip-10-0-139-173 kubenswrapper[2551]: W0526 03:03:59.583253    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod9ccf5aec_e227_435b_9629_f2d716c329f3.slice/crio-a1476b8b73fcb76c742f3ec96b8791c7f25b3526d620aee60569740384aaf3f4 WatchSource:0}: Error finding container a1476b8b73fcb76c742f3ec96b8791c7f25b3526d620aee60569740384aaf3f4: Status 404 returned error can't find the container with id a1476b8b73fcb76c742f3ec96b8791c7f25b3526d620aee60569740384aaf3f4
May 26 03:04:00.501043 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:00.501010    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerStarted","Data":"4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39"}
May 26 03:04:00.501043 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:00.501045    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerStarted","Data":"a1476b8b73fcb76c742f3ec96b8791c7f25b3526d620aee60569740384aaf3f4"}
May 26 03:04:04.515332 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:04.515237    2551 generic.go:358] "Generic (PLEG): container finished" podID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerID="4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39" exitCode=0
May 26 03:04:04.515332 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:04.515312    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerDied","Data":"4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39"}
May 26 03:04:05.520323 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:05.520287    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerStarted","Data":"9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc"}
May 26 03:04:06.283125 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:06.283075    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.40:8443: connect: connection refused"
May 26 03:04:06.311680 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:06.311640    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.40:6567/health\": dial tcp 10.134.0.40:6567: connect: connection refused"
May 26 03:04:10.538043 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:10.538008    2551 generic.go:358] "Generic (PLEG): container finished" podID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerID="9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc" exitCode=0
May 26 03:04:10.538528 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:10.538065    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerDied","Data":"9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc"}
May 26 03:04:11.545564 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:11.545520    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerStarted","Data":"b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b"}
May 26 03:04:11.546098 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:11.545572    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerStarted","Data":"c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1"}
May 26 03:04:11.546098 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:11.545587    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerStarted","Data":"24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3"}
May 26 03:04:11.546098 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:11.545601    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerStarted","Data":"c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a"}
May 26 03:04:11.568329 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:11.568252    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podStartSLOduration=13.56823305 podStartE2EDuration="13.56823305s" podCreationTimestamp="2026-05-26 03:03:58 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 03:04:11.566208421 +0000 UTC m=+1689.122997094" watchObservedRunningTime="2026-05-26 03:04:11.56823305 +0000 UTC m=+1689.125021719"
May 26 03:04:14.454826 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:14.454784    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:14.455379 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:14.455360    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:14.455494 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:14.455483    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:14.455894 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:14.455862    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:14.457337 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:14.457182    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.41:8443: connect: connection refused"
May 26 03:04:14.457337 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:14.457211    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.41:6571: connect: connection refused"
May 26 03:04:14.457337 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:14.457218    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.41:8816: connect: connection refused"
May 26 03:04:14.457337 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:14.457267    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.41:6567/health\": dial tcp 10.134.0.41:6567: connect: connection refused"
May 26 03:04:16.283107 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:16.283013    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.40:8443: connect: connection refused"
May 26 03:04:16.311977 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:16.311942    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.40:6567/health\": dial tcp 10.134.0.40:6567: connect: connection refused"
May 26 03:04:17.455211 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:17.455175    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:17.455641 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:17.455357    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:17.455641 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:17.455636    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:17.456034 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:17.455998    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:17.456143 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:17.456055    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:17.456143 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:17.456070    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:17.456143 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:17.456117    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:17.456143 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:17.456146    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:17.456351 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:17.456174    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:17.459794 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:17.459774    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:17.566718 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:17.566682    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:17.569408 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:17.569387    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:04:26.282752 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:26.282707    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.40:8443: connect: connection refused"
May 26 03:04:26.283167 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:26.282821    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:04:26.311476 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:26.311446    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.40:6567/health\": dial tcp 10.134.0.40:6567: connect: connection refused"
May 26 03:04:26.311659 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:26.311567    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:04:28.602513 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:28.602417    2551 generic.go:358] "Generic (PLEG): container finished" podID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerID="f99470d4b0113e15614dc88625d4eaa2e61cef38ba84aabea1c5ff26e2faf9b3" exitCode=137
May 26 03:04:28.602513 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:28.602443    2551 generic.go:358] "Generic (PLEG): container finished" podID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerID="4787928d8d027f195192550559d62afd14519a9d4f26741804035a45545a27b5" exitCode=137
May 26 03:04:28.602513 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:28.602490    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerDied","Data":"f99470d4b0113e15614dc88625d4eaa2e61cef38ba84aabea1c5ff26e2faf9b3"}
May 26 03:04:28.602955 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:28.602531    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerDied","Data":"4787928d8d027f195192550559d62afd14519a9d4f26741804035a45545a27b5"}
May 26 03:04:28.940691 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:28.940670    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:04:29.130211 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.130137    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-registry-tls\") pod \"ab722eb4-e466-4691-9b30-27275d57b2c3\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") "
May 26 03:04:29.130211 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.130168    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-online-tls\") pod \"ab722eb4-e466-4691-9b30-27275d57b2c3\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") "
May 26 03:04:29.130211 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.130197    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab722eb4-e466-4691-9b30-27275d57b2c3-feast-data\") pod \"ab722eb4-e466-4691-9b30-27275d57b2c3\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") "
May 26 03:04:29.130493 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.130216    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-offline-tls\") pod \"ab722eb4-e466-4691-9b30-27275d57b2c3\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") "
May 26 03:04:29.130493 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.130233    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-czmdn\" (UniqueName: \"kubernetes.io/projected/ab722eb4-e466-4691-9b30-27275d57b2c3-kube-api-access-czmdn\") pod \"ab722eb4-e466-4691-9b30-27275d57b2c3\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") "
May 26 03:04:29.130493 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.130322    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-ui-tls\") pod \"ab722eb4-e466-4691-9b30-27275d57b2c3\" (UID: \"ab722eb4-e466-4691-9b30-27275d57b2c3\") "
May 26 03:04:29.130817 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.130788    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ab722eb4-e466-4691-9b30-27275d57b2c3-feast-data" (OuterVolumeSpecName: "feast-data") pod "ab722eb4-e466-4691-9b30-27275d57b2c3" (UID: "ab722eb4-e466-4691-9b30-27275d57b2c3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 03:04:29.132573 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.132520    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ab722eb4-e466-4691-9b30-27275d57b2c3" (UID: "ab722eb4-e466-4691-9b30-27275d57b2c3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:04:29.132679 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.132595    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ab722eb4-e466-4691-9b30-27275d57b2c3" (UID: "ab722eb4-e466-4691-9b30-27275d57b2c3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:04:29.132679 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.132610    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ab722eb4-e466-4691-9b30-27275d57b2c3-kube-api-access-czmdn" (OuterVolumeSpecName: "kube-api-access-czmdn") pod "ab722eb4-e466-4691-9b30-27275d57b2c3" (UID: "ab722eb4-e466-4691-9b30-27275d57b2c3"). InnerVolumeSpecName "kube-api-access-czmdn". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 03:04:29.132762 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.132709    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "ab722eb4-e466-4691-9b30-27275d57b2c3" (UID: "ab722eb4-e466-4691-9b30-27275d57b2c3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:04:29.132762 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.132733    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-online-tls" (OuterVolumeSpecName: "online-tls") pod "ab722eb4-e466-4691-9b30-27275d57b2c3" (UID: "ab722eb4-e466-4691-9b30-27275d57b2c3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:04:29.231796 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.231766    2551 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-ui-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:04:29.231796 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.231797    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:04:29.232010 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.231812    2551 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-online-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:04:29.232010 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.231823    2551 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ab722eb4-e466-4691-9b30-27275d57b2c3-feast-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:04:29.232010 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.231834    2551 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ab722eb4-e466-4691-9b30-27275d57b2c3-offline-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:04:29.232010 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.231846    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-czmdn\" (UniqueName: \"kubernetes.io/projected/ab722eb4-e466-4691-9b30-27275d57b2c3-kube-api-access-czmdn\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:04:29.608317 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.608280    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg" event={"ID":"ab722eb4-e466-4691-9b30-27275d57b2c3","Type":"ContainerDied","Data":"7cdfecb391119940b56ef8388695808470a4354243567325dbf91fe8c475599c"}
May 26 03:04:29.608707 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.608335    2551 scope.go:117] "RemoveContainer" containerID="4fdbec5c96e1b68137b440567c57490b51f4c1c8f0fb005161286a40f39e1f60"
May 26 03:04:29.608707 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.608298    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"
May 26 03:04:29.617704 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.617686    2551 scope.go:117] "RemoveContainer" containerID="f99470d4b0113e15614dc88625d4eaa2e61cef38ba84aabea1c5ff26e2faf9b3"
May 26 03:04:29.624710 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.624694    2551 scope.go:117] "RemoveContainer" containerID="9cdbb95818d33d2d96d0d52e1855181833855e4b25e64881f9f93d178d37a50b"
May 26 03:04:29.630235 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.630200    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"]
May 26 03:04:29.632404 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.632384    2551 scope.go:117] "RemoveContainer" containerID="4787928d8d027f195192550559d62afd14519a9d4f26741804035a45545a27b5"
May 26 03:04:29.634991 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.634965    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-fd4bg"]
May 26 03:04:29.639724 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.639699    2551 scope.go:117] "RemoveContainer" containerID="8243d027bb7c64c59ef23b03bef5907e00da9c21f813fc8a789c2044807885f5"
May 26 03:04:29.646620 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:29.646605    2551 scope.go:117] "RemoveContainer" containerID="66ab5182af638c8ce3e223b072e84229b8926bd4d41c29408976c24a6aba5fb2"
May 26 03:04:31.044757 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:04:31.044724    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" path="/var/lib/kubelet/pods/ab722eb4-e466-4691-9b30-27275d57b2c3/volumes"
May 26 03:05:19.985062 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985023    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"]
May 26 03:05:19.985477 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985447    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="registry"
May 26 03:05:19.985477 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985461    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="registry"
May 26 03:05:19.985590 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985483    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="offline"
May 26 03:05:19.985590 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985492    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="offline"
May 26 03:05:19.985590 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985541    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="feast-apply"
May 26 03:05:19.985590 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985550    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="feast-apply"
May 26 03:05:19.985590 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985569    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="feast-init"
May 26 03:05:19.985590 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985577    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="feast-init"
May 26 03:05:19.985590 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985587    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="online"
May 26 03:05:19.985833 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985595    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="online"
May 26 03:05:19.985833 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985608    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="ui"
May 26 03:05:19.985833 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985616    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="ui"
May 26 03:05:19.985833 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985681    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="registry"
May 26 03:05:19.985833 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985694    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="online"
May 26 03:05:19.985833 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985703    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="ui"
May 26 03:05:19.985833 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.985715    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="ab722eb4-e466-4691-9b30-27275d57b2c3" containerName="offline"
May 26 03:05:19.990684 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.990664    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:19.993021 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.992999    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 26 03:05:19.993178 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.993038    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 26 03:05:19.993259 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.993059    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 26 03:05:19.993737 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.993715    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 26 03:05:19.993856 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.993754    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 26 03:05:19.993856 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.993718    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 26 03:05:19.993856 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.993810    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-zzmq8\""
May 26 03:05:19.996456 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:19.996436    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"]
May 26 03:05:20.138332 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.138297    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.138332 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.138333    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bdfc82db-37c6-400f-ab72-07aa3ef660a2-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.138517 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.138360    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.138517 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.138378    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.138517 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.138453    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bdfc82db-37c6-400f-ab72-07aa3ef660a2-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.138517 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.138482    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-m8zg7\" (UniqueName: \"kubernetes.io/projected/bdfc82db-37c6-400f-ab72-07aa3ef660a2-kube-api-access-m8zg7\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.239309 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.239235    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.239309 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.239272    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bdfc82db-37c6-400f-ab72-07aa3ef660a2-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.239309 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.239308    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.239547 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.239328    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.239547 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.239363    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bdfc82db-37c6-400f-ab72-07aa3ef660a2-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.239547 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.239384    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-m8zg7\" (UniqueName: \"kubernetes.io/projected/bdfc82db-37c6-400f-ab72-07aa3ef660a2-kube-api-access-m8zg7\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.239807 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.239784    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bdfc82db-37c6-400f-ab72-07aa3ef660a2-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.240112 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.240089    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bdfc82db-37c6-400f-ab72-07aa3ef660a2-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.241865 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.241844    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.241977 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.241864    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.241977 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.241891    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.246439 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.246418    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-m8zg7\" (UniqueName: \"kubernetes.io/projected/bdfc82db-37c6-400f-ab72-07aa3ef660a2-kube-api-access-m8zg7\") pod \"feast-simple-feast-remote-setup-5cff94746b-jbsvm\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.302529 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.302501    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:20.435070 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.435037    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"]
May 26 03:05:20.438215 ip-10-0-139-173 kubenswrapper[2551]: W0526 03:05:20.438187    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podbdfc82db_37c6_400f_ab72_07aa3ef660a2.slice/crio-07557f9ccb5b6df932e65b32ce540a66fe1e8dc315ab9fdec0a88a6fdc5ef4b4 WatchSource:0}: Error finding container 07557f9ccb5b6df932e65b32ce540a66fe1e8dc315ab9fdec0a88a6fdc5ef4b4: Status 404 returned error can't find the container with id 07557f9ccb5b6df932e65b32ce540a66fe1e8dc315ab9fdec0a88a6fdc5ef4b4
May 26 03:05:20.440090 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.440073    2551 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 26 03:05:20.761300 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.761268    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" event={"ID":"bdfc82db-37c6-400f-ab72-07aa3ef660a2","Type":"ContainerStarted","Data":"d7609f70839f5e4f85048652686c64ba20732a0ef6ae638f64379418625968f9"}
May 26 03:05:20.761300 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:20.761301    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" event={"ID":"bdfc82db-37c6-400f-ab72-07aa3ef660a2","Type":"ContainerStarted","Data":"07557f9ccb5b6df932e65b32ce540a66fe1e8dc315ab9fdec0a88a6fdc5ef4b4"}
May 26 03:05:25.778087 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:25.778052    2551 generic.go:358] "Generic (PLEG): container finished" podID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerID="d7609f70839f5e4f85048652686c64ba20732a0ef6ae638f64379418625968f9" exitCode=0
May 26 03:05:25.778471 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:25.778126    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" event={"ID":"bdfc82db-37c6-400f-ab72-07aa3ef660a2","Type":"ContainerDied","Data":"d7609f70839f5e4f85048652686c64ba20732a0ef6ae638f64379418625968f9"}
May 26 03:05:26.783661 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:26.783625    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" event={"ID":"bdfc82db-37c6-400f-ab72-07aa3ef660a2","Type":"ContainerStarted","Data":"2b519f9ad8d4637ad604b731a676fd3d5f8d16d3589cd4d281eab1d85c936601"}
May 26 03:05:31.803184 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:31.803149    2551 generic.go:358] "Generic (PLEG): container finished" podID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerID="2b519f9ad8d4637ad604b731a676fd3d5f8d16d3589cd4d281eab1d85c936601" exitCode=0
May 26 03:05:31.803550 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:31.803224    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" event={"ID":"bdfc82db-37c6-400f-ab72-07aa3ef660a2","Type":"ContainerDied","Data":"2b519f9ad8d4637ad604b731a676fd3d5f8d16d3589cd4d281eab1d85c936601"}
May 26 03:05:32.810213 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:32.810168    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" event={"ID":"bdfc82db-37c6-400f-ab72-07aa3ef660a2","Type":"ContainerStarted","Data":"5857e06c69602b32b57320f712f38205cfc370b48b650f609601008e7b0d35c2"}
May 26 03:05:32.810729 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:32.810221    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" event={"ID":"bdfc82db-37c6-400f-ab72-07aa3ef660a2","Type":"ContainerStarted","Data":"d001fe31c2c1736271e06e3709e0589aa7982be1da153ead66253f986d6f533a"}
May 26 03:05:32.810729 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:32.810237    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" event={"ID":"bdfc82db-37c6-400f-ab72-07aa3ef660a2","Type":"ContainerStarted","Data":"ca2586702f4dee2cb704894c0c55de6ad5bbc3a211e1f922d65ef0e7cf8b4cac"}
May 26 03:05:32.832807 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:32.832717    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podStartSLOduration=13.832696126 podStartE2EDuration="13.832696126s" podCreationTimestamp="2026-05-26 03:05:19 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 03:05:32.829339053 +0000 UTC m=+1770.386127769" watchObservedRunningTime="2026-05-26 03:05:32.832696126 +0000 UTC m=+1770.389484795"
May 26 03:05:35.303225 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:35.303182    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:35.303225 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:35.303235    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:35.303939 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:35.303248    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:35.304843 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:35.304803    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.42:8816: connect: connection refused"
May 26 03:05:35.304985 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:35.304841    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.42:8443: connect: connection refused"
May 26 03:05:35.304985 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:35.304852    2551 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.42:6567/health\": dial tcp 10.134.0.42:6567: connect: connection refused"
May 26 03:05:38.304155 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:38.304126    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:38.304653 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:38.304624    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:38.304791 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:38.304661    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:38.304868 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:38.304855    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:38.308497 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:38.308476    2551 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:38.830731 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:38.830706    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:38.830731 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:38.830738    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:38.831077 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:38.831052    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:05:38.833664 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:05:38.833639    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:06:03.050618 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:03.050586    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 03:06:03.052161 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:03.052141    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 03:06:03.054807 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:03.054789    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 03:06:03.056239 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:03.056224    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 03:06:40.997174 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:40.997138    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"]
May 26 03:06:40.998087 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:40.998028    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="online" containerID="cri-o://ca2586702f4dee2cb704894c0c55de6ad5bbc3a211e1f922d65ef0e7cf8b4cac" gracePeriod=30
May 26 03:06:40.998087 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:40.998049    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="ui" containerID="cri-o://5857e06c69602b32b57320f712f38205cfc370b48b650f609601008e7b0d35c2" gracePeriod=30
May 26 03:06:40.998087 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:40.998075    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="offline" containerID="cri-o://d001fe31c2c1736271e06e3709e0589aa7982be1da153ead66253f986d6f533a" gracePeriod=30
May 26 03:06:41.158398 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:41.158358    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"]
May 26 03:06:41.159035 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:41.158846    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="registry" containerID="cri-o://c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a" gracePeriod=30
May 26 03:06:41.159035 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:41.158871    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="ui" containerID="cri-o://b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b" gracePeriod=30
May 26 03:06:41.159035 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:41.158891    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="online" containerID="cri-o://24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3" gracePeriod=30
May 26 03:06:41.159035 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:41.158871    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="offline" containerID="cri-o://c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1" gracePeriod=30
May 26 03:06:42.025099 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:42.025067    2551 generic.go:358] "Generic (PLEG): container finished" podID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerID="b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b" exitCode=0
May 26 03:06:42.025099 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:42.025094    2551 generic.go:358] "Generic (PLEG): container finished" podID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerID="24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3" exitCode=0
May 26 03:06:42.025572 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:42.025154    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerDied","Data":"b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b"}
May 26 03:06:42.025572 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:42.025195    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerDied","Data":"24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3"}
May 26 03:06:42.027266 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:42.027242    2551 generic.go:358] "Generic (PLEG): container finished" podID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerID="5857e06c69602b32b57320f712f38205cfc370b48b650f609601008e7b0d35c2" exitCode=0
May 26 03:06:42.027266 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:42.027260    2551 generic.go:358] "Generic (PLEG): container finished" podID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerID="ca2586702f4dee2cb704894c0c55de6ad5bbc3a211e1f922d65ef0e7cf8b4cac" exitCode=0
May 26 03:06:42.027417 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:42.027277    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" event={"ID":"bdfc82db-37c6-400f-ab72-07aa3ef660a2","Type":"ContainerDied","Data":"5857e06c69602b32b57320f712f38205cfc370b48b650f609601008e7b0d35c2"}
May 26 03:06:42.027417 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:42.027294    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" event={"ID":"bdfc82db-37c6-400f-ab72-07aa3ef660a2","Type":"ContainerDied","Data":"ca2586702f4dee2cb704894c0c55de6ad5bbc3a211e1f922d65ef0e7cf8b4cac"}
May 26 03:06:47.456719 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:47.456667    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.41:8443: connect: connection refused"
May 26 03:06:47.567651 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:47.567612    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.41:6567/health\": dial tcp 10.134.0.41:6567: connect: connection refused"
May 26 03:06:48.305663 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:48.305620    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.42:8443: connect: connection refused"
May 26 03:06:48.831108 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:48.831045    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.42:6567/health\": dial tcp 10.134.0.42:6567: connect: connection refused"
May 26 03:06:57.455933 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:57.455857    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.41:8443: connect: connection refused"
May 26 03:06:57.567590 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:57.567541    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.41:6567/health\": dial tcp 10.134.0.41:6567: connect: connection refused"
May 26 03:06:58.304782 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:58.304740    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.42:8443: connect: connection refused"
May 26 03:06:58.831138 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:06:58.831092    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.42:6567/health\": dial tcp 10.134.0.42:6567: connect: connection refused"
May 26 03:07:07.456776 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:07.456727    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.41:8443: connect: connection refused"
May 26 03:07:07.457202 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:07.456870    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:07:07.567256 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:07.567220    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.41:6567/health\": dial tcp 10.134.0.41:6567: connect: connection refused"
May 26 03:07:07.567425 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:07.567328    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:07:08.304938 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:08.304882    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.42:8443: connect: connection refused"
May 26 03:07:08.305145 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:08.305020    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:07:08.831688 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:08.831646    2551 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="online" probeResult="failure" output="Get \"https://10.134.0.42:6567/health\": dial tcp 10.134.0.42:6567: connect: connection refused"
May 26 03:07:08.832148 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:08.831776    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:07:11.114174 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.114142    2551 generic.go:358] "Generic (PLEG): container finished" podID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerID="d001fe31c2c1736271e06e3709e0589aa7982be1da153ead66253f986d6f533a" exitCode=137
May 26 03:07:11.114540 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.114213    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" event={"ID":"bdfc82db-37c6-400f-ab72-07aa3ef660a2","Type":"ContainerDied","Data":"d001fe31c2c1736271e06e3709e0589aa7982be1da153ead66253f986d6f533a"}
May 26 03:07:11.182419 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.182393    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:07:11.215083 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.215050    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-ui-tls\") pod \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") "
May 26 03:07:11.215262 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.215098    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-online-tls\") pod \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") "
May 26 03:07:11.215262 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.215121    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bdfc82db-37c6-400f-ab72-07aa3ef660a2-feast-data\") pod \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") "
May 26 03:07:11.215262 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.215151    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-offline-tls\") pod \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") "
May 26 03:07:11.215262 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.215243    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-m8zg7\" (UniqueName: \"kubernetes.io/projected/bdfc82db-37c6-400f-ab72-07aa3ef660a2-kube-api-access-m8zg7\") pod \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") "
May 26 03:07:11.215582 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.215285    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bdfc82db-37c6-400f-ab72-07aa3ef660a2-registry-tls\") pod \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\" (UID: \"bdfc82db-37c6-400f-ab72-07aa3ef660a2\") "
May 26 03:07:11.215640 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.215597    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/bdfc82db-37c6-400f-ab72-07aa3ef660a2-feast-data" (OuterVolumeSpecName: "feast-data") pod "bdfc82db-37c6-400f-ab72-07aa3ef660a2" (UID: "bdfc82db-37c6-400f-ab72-07aa3ef660a2"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 03:07:11.215834 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.215801    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/bdfc82db-37c6-400f-ab72-07aa3ef660a2-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "bdfc82db-37c6-400f-ab72-07aa3ef660a2" (UID: "bdfc82db-37c6-400f-ab72-07aa3ef660a2"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 26 03:07:11.217571 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.217543    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "bdfc82db-37c6-400f-ab72-07aa3ef660a2" (UID: "bdfc82db-37c6-400f-ab72-07aa3ef660a2"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:07:11.217751 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.217731    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "bdfc82db-37c6-400f-ab72-07aa3ef660a2" (UID: "bdfc82db-37c6-400f-ab72-07aa3ef660a2"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:07:11.217811 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.217733    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/bdfc82db-37c6-400f-ab72-07aa3ef660a2-kube-api-access-m8zg7" (OuterVolumeSpecName: "kube-api-access-m8zg7") pod "bdfc82db-37c6-400f-ab72-07aa3ef660a2" (UID: "bdfc82db-37c6-400f-ab72-07aa3ef660a2"). InnerVolumeSpecName "kube-api-access-m8zg7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 03:07:11.217811 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.217775    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-online-tls" (OuterVolumeSpecName: "online-tls") pod "bdfc82db-37c6-400f-ab72-07aa3ef660a2" (UID: "bdfc82db-37c6-400f-ab72-07aa3ef660a2"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:07:11.315947 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.315891    2551 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-offline-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:11.315947 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.315946    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-m8zg7\" (UniqueName: \"kubernetes.io/projected/bdfc82db-37c6-400f-ab72-07aa3ef660a2-kube-api-access-m8zg7\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:11.315947 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.315955    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/bdfc82db-37c6-400f-ab72-07aa3ef660a2-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:11.316168 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.315964    2551 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-ui-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:11.316168 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.315973    2551 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/bdfc82db-37c6-400f-ab72-07aa3ef660a2-online-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:11.316168 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.315981    2551 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/bdfc82db-37c6-400f-ab72-07aa3ef660a2-feast-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:11.809408 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.809387    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:07:11.921475 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.921435    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-ui-tls\") pod \"9ccf5aec-e227-435b-9629-f2d716c329f3\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") "
May 26 03:07:11.921684 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.921498    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-offline-tls\") pod \"9ccf5aec-e227-435b-9629-f2d716c329f3\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") "
May 26 03:07:11.921684 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.921529    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5aec-e227-435b-9629-f2d716c329f3-feast-data\") pod \"9ccf5aec-e227-435b-9629-f2d716c329f3\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") "
May 26 03:07:11.921684 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.921558    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-registry-tls\") pod \"9ccf5aec-e227-435b-9629-f2d716c329f3\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") "
May 26 03:07:11.921863 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.921730    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-online-tls\") pod \"9ccf5aec-e227-435b-9629-f2d716c329f3\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") "
May 26 03:07:11.921863 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.921783    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-t2hpp\" (UniqueName: \"kubernetes.io/projected/9ccf5aec-e227-435b-9629-f2d716c329f3-kube-api-access-t2hpp\") pod \"9ccf5aec-e227-435b-9629-f2d716c329f3\" (UID: \"9ccf5aec-e227-435b-9629-f2d716c329f3\") "
May 26 03:07:11.922144 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.922098    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/9ccf5aec-e227-435b-9629-f2d716c329f3-feast-data" (OuterVolumeSpecName: "feast-data") pod "9ccf5aec-e227-435b-9629-f2d716c329f3" (UID: "9ccf5aec-e227-435b-9629-f2d716c329f3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 03:07:11.923882 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.923849    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "9ccf5aec-e227-435b-9629-f2d716c329f3" (UID: "9ccf5aec-e227-435b-9629-f2d716c329f3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:07:11.924005 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.923877    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-online-tls" (OuterVolumeSpecName: "online-tls") pod "9ccf5aec-e227-435b-9629-f2d716c329f3" (UID: "9ccf5aec-e227-435b-9629-f2d716c329f3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:07:11.924005 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.923970    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "9ccf5aec-e227-435b-9629-f2d716c329f3" (UID: "9ccf5aec-e227-435b-9629-f2d716c329f3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:07:11.924120 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.924101    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/9ccf5aec-e227-435b-9629-f2d716c329f3-kube-api-access-t2hpp" (OuterVolumeSpecName: "kube-api-access-t2hpp") pod "9ccf5aec-e227-435b-9629-f2d716c329f3" (UID: "9ccf5aec-e227-435b-9629-f2d716c329f3"). InnerVolumeSpecName "kube-api-access-t2hpp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 03:07:11.924169 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:11.924101    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "9ccf5aec-e227-435b-9629-f2d716c329f3" (UID: "9ccf5aec-e227-435b-9629-f2d716c329f3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 26 03:07:12.022688 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.022659    2551 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-registry-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:12.022688 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.022689    2551 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-online-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:12.022847 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.022699    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-t2hpp\" (UniqueName: \"kubernetes.io/projected/9ccf5aec-e227-435b-9629-f2d716c329f3-kube-api-access-t2hpp\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:12.022847 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.022709    2551 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-ui-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:12.022847 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.022720    2551 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/9ccf5aec-e227-435b-9629-f2d716c329f3-offline-tls\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:12.022847 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.022729    2551 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/9ccf5aec-e227-435b-9629-f2d716c329f3-feast-data\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:12.119991 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.119965    2551 generic.go:358] "Generic (PLEG): container finished" podID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerID="c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1" exitCode=137
May 26 03:07:12.119991 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.119986    2551 generic.go:358] "Generic (PLEG): container finished" podID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerID="c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a" exitCode=137
May 26 03:07:12.120538 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.120034    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"
May 26 03:07:12.120538 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.120057    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerDied","Data":"c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1"}
May 26 03:07:12.120538 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.120107    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerDied","Data":"c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a"}
May 26 03:07:12.120538 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.120126    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc" event={"ID":"9ccf5aec-e227-435b-9629-f2d716c329f3","Type":"ContainerDied","Data":"a1476b8b73fcb76c742f3ec96b8791c7f25b3526d620aee60569740384aaf3f4"}
May 26 03:07:12.120538 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.120149    2551 scope.go:117] "RemoveContainer" containerID="b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b"
May 26 03:07:12.122543 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.122516    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm" event={"ID":"bdfc82db-37c6-400f-ab72-07aa3ef660a2","Type":"ContainerDied","Data":"07557f9ccb5b6df932e65b32ce540a66fe1e8dc315ab9fdec0a88a6fdc5ef4b4"}
May 26 03:07:12.122645 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.122557    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"
May 26 03:07:12.131490 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.131472    2551 scope.go:117] "RemoveContainer" containerID="c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1"
May 26 03:07:12.138556 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.138537    2551 scope.go:117] "RemoveContainer" containerID="24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3"
May 26 03:07:12.141810 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.141786    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"]
May 26 03:07:12.145800 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.145784    2551 scope.go:117] "RemoveContainer" containerID="c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a"
May 26 03:07:12.149544 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.149512    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-qlntc"]
May 26 03:07:12.153122 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.153105    2551 scope.go:117] "RemoveContainer" containerID="9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc"
May 26 03:07:12.158911 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.158873    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"]
May 26 03:07:12.160323 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.160308    2551 scope.go:117] "RemoveContainer" containerID="4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39"
May 26 03:07:12.162357 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.162336    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-jbsvm"]
May 26 03:07:12.167360 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.167345    2551 scope.go:117] "RemoveContainer" containerID="b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b"
May 26 03:07:12.167632 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:07:12.167614    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b\": container with ID starting with b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b not found: ID does not exist" containerID="b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b"
May 26 03:07:12.167679 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.167643    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b"} err="failed to get container status \"b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b\": rpc error: code = NotFound desc = could not find container \"b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b\": container with ID starting with b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b not found: ID does not exist"
May 26 03:07:12.167679 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.167663    2551 scope.go:117] "RemoveContainer" containerID="c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1"
May 26 03:07:12.167889 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:07:12.167871    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1\": container with ID starting with c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1 not found: ID does not exist" containerID="c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1"
May 26 03:07:12.167945 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.167910    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1"} err="failed to get container status \"c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1\": rpc error: code = NotFound desc = could not find container \"c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1\": container with ID starting with c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1 not found: ID does not exist"
May 26 03:07:12.167945 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.167929    2551 scope.go:117] "RemoveContainer" containerID="24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3"
May 26 03:07:12.168134 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:07:12.168116    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3\": container with ID starting with 24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3 not found: ID does not exist" containerID="24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3"
May 26 03:07:12.168191 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.168142    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3"} err="failed to get container status \"24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3\": rpc error: code = NotFound desc = could not find container \"24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3\": container with ID starting with 24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3 not found: ID does not exist"
May 26 03:07:12.168191 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.168167    2551 scope.go:117] "RemoveContainer" containerID="c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a"
May 26 03:07:12.168383 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:07:12.168368    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a\": container with ID starting with c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a not found: ID does not exist" containerID="c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a"
May 26 03:07:12.168426 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.168386    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a"} err="failed to get container status \"c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a\": rpc error: code = NotFound desc = could not find container \"c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a\": container with ID starting with c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a not found: ID does not exist"
May 26 03:07:12.168426 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.168400    2551 scope.go:117] "RemoveContainer" containerID="9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc"
May 26 03:07:12.168598 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:07:12.168581    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc\": container with ID starting with 9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc not found: ID does not exist" containerID="9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc"
May 26 03:07:12.168638 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.168600    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc"} err="failed to get container status \"9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc\": rpc error: code = NotFound desc = could not find container \"9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc\": container with ID starting with 9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc not found: ID does not exist"
May 26 03:07:12.168638 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.168612    2551 scope.go:117] "RemoveContainer" containerID="4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39"
May 26 03:07:12.168774 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:07:12.168761    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39\": container with ID starting with 4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39 not found: ID does not exist" containerID="4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39"
May 26 03:07:12.168815 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.168779    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39"} err="failed to get container status \"4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39\": rpc error: code = NotFound desc = could not find container \"4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39\": container with ID starting with 4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39 not found: ID does not exist"
May 26 03:07:12.168815 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.168794    2551 scope.go:117] "RemoveContainer" containerID="b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b"
May 26 03:07:12.168999 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.168985    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b"} err="failed to get container status \"b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b\": rpc error: code = NotFound desc = could not find container \"b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b\": container with ID starting with b5442cf6d5a3b00f48a8de0c06f16cee971f8ef269fd8c7ca31d2d1fe302fe3b not found: ID does not exist"
May 26 03:07:12.169045 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.168998    2551 scope.go:117] "RemoveContainer" containerID="c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1"
May 26 03:07:12.169189 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.169174    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1"} err="failed to get container status \"c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1\": rpc error: code = NotFound desc = could not find container \"c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1\": container with ID starting with c231542b43081834ce978069d96b7ecdb5f02ff7e326e621963b3b8ab6faeda1 not found: ID does not exist"
May 26 03:07:12.169251 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.169189    2551 scope.go:117] "RemoveContainer" containerID="24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3"
May 26 03:07:12.169375 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.169355    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3"} err="failed to get container status \"24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3\": rpc error: code = NotFound desc = could not find container \"24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3\": container with ID starting with 24a94767017a5467dd69ea7880b7b2f11c5e1ba5fe241e3cf23868f0aa957aa3 not found: ID does not exist"
May 26 03:07:12.169419 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.169375    2551 scope.go:117] "RemoveContainer" containerID="c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a"
May 26 03:07:12.169534 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.169514    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a"} err="failed to get container status \"c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a\": rpc error: code = NotFound desc = could not find container \"c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a\": container with ID starting with c4f6378c6bcc4dccf6cb9b42a1cc811c79b0d79e159629edee21825f2e56542a not found: ID does not exist"
May 26 03:07:12.169584 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.169536    2551 scope.go:117] "RemoveContainer" containerID="9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc"
May 26 03:07:12.169734 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.169714    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc"} err="failed to get container status \"9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc\": rpc error: code = NotFound desc = could not find container \"9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc\": container with ID starting with 9b5ab294c3e80ebcb1b796d2a7e208270b5ff567b673ecb8d0efada952c4bddc not found: ID does not exist"
May 26 03:07:12.169782 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.169736    2551 scope.go:117] "RemoveContainer" containerID="4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39"
May 26 03:07:12.169970 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.169954    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39"} err="failed to get container status \"4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39\": rpc error: code = NotFound desc = could not find container \"4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39\": container with ID starting with 4443ad0b9e2f048e8bf88ed963d82777968118b23f765b3f6e46bee3789e0e39 not found: ID does not exist"
May 26 03:07:12.170022 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.169970    2551 scope.go:117] "RemoveContainer" containerID="5857e06c69602b32b57320f712f38205cfc370b48b650f609601008e7b0d35c2"
May 26 03:07:12.176787 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.176772    2551 scope.go:117] "RemoveContainer" containerID="d001fe31c2c1736271e06e3709e0589aa7982be1da153ead66253f986d6f533a"
May 26 03:07:12.183389 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.183373    2551 scope.go:117] "RemoveContainer" containerID="ca2586702f4dee2cb704894c0c55de6ad5bbc3a211e1f922d65ef0e7cf8b4cac"
May 26 03:07:12.189991 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.189968    2551 scope.go:117] "RemoveContainer" containerID="2b519f9ad8d4637ad604b731a676fd3d5f8d16d3589cd4d281eab1d85c936601"
May 26 03:07:12.196494 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:12.196480    2551 scope.go:117] "RemoveContainer" containerID="d7609f70839f5e4f85048652686c64ba20732a0ef6ae638f64379418625968f9"
May 26 03:07:13.044739 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:13.044695    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" path="/var/lib/kubelet/pods/9ccf5aec-e227-435b-9629-f2d716c329f3/volumes"
May 26 03:07:13.045421 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:13.045399    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" path="/var/lib/kubelet/pods/bdfc82db-37c6-400f-ab72-07aa3ef660a2/volumes"
May 26 03:07:30.071593 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071555    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-rwtsd/must-gather-fwh9z"]
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071879    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="ui"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071889    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="ui"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071914    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="online"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071921    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="online"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071928    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="ui"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071934    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="ui"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071949    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="feast-apply"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071954    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="feast-apply"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071963    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="feast-init"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071968    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="feast-init"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071974    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="offline"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071978    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="offline"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071986    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="feast-apply"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071990    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="feast-apply"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.071997    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="registry"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072002    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="registry"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072009    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="online"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072014    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="online"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072021    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="feast-init"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072026    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="feast-init"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072031    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="offline"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072036    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="offline"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072080    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="offline"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072088    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="registry"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072094    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="offline"
May 26 03:07:30.072092 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072103    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="online"
May 26 03:07:30.072893 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072110    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="bdfc82db-37c6-400f-ab72-07aa3ef660a2" containerName="ui"
May 26 03:07:30.072893 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072118    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="online"
May 26 03:07:30.072893 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.072124    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="9ccf5aec-e227-435b-9629-f2d716c329f3" containerName="ui"
May 26 03:07:30.075077 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.075059    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rwtsd/must-gather-fwh9z"
May 26 03:07:30.077276 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.077249    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rwtsd\"/\"openshift-service-ca.crt\""
May 26 03:07:30.077416 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.077313    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-rwtsd\"/\"default-dockercfg-r988x\""
May 26 03:07:30.077416 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.077333    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-rwtsd\"/\"kube-root-ca.crt\""
May 26 03:07:30.083498 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.083472    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rwtsd/must-gather-fwh9z"]
May 26 03:07:30.169178 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.169139    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-wn9n4\" (UniqueName: \"kubernetes.io/projected/7eb6f778-95ea-4aaf-8500-febf6b348523-kube-api-access-wn9n4\") pod \"must-gather-fwh9z\" (UID: \"7eb6f778-95ea-4aaf-8500-febf6b348523\") " pod="openshift-must-gather-rwtsd/must-gather-fwh9z"
May 26 03:07:30.169366 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.169225    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7eb6f778-95ea-4aaf-8500-febf6b348523-must-gather-output\") pod \"must-gather-fwh9z\" (UID: \"7eb6f778-95ea-4aaf-8500-febf6b348523\") " pod="openshift-must-gather-rwtsd/must-gather-fwh9z"
May 26 03:07:30.270111 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.270069    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7eb6f778-95ea-4aaf-8500-febf6b348523-must-gather-output\") pod \"must-gather-fwh9z\" (UID: \"7eb6f778-95ea-4aaf-8500-febf6b348523\") " pod="openshift-must-gather-rwtsd/must-gather-fwh9z"
May 26 03:07:30.270289 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.270125    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-wn9n4\" (UniqueName: \"kubernetes.io/projected/7eb6f778-95ea-4aaf-8500-febf6b348523-kube-api-access-wn9n4\") pod \"must-gather-fwh9z\" (UID: \"7eb6f778-95ea-4aaf-8500-febf6b348523\") " pod="openshift-must-gather-rwtsd/must-gather-fwh9z"
May 26 03:07:30.270418 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.270396    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7eb6f778-95ea-4aaf-8500-febf6b348523-must-gather-output\") pod \"must-gather-fwh9z\" (UID: \"7eb6f778-95ea-4aaf-8500-febf6b348523\") " pod="openshift-must-gather-rwtsd/must-gather-fwh9z"
May 26 03:07:30.277379 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.277350    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-wn9n4\" (UniqueName: \"kubernetes.io/projected/7eb6f778-95ea-4aaf-8500-febf6b348523-kube-api-access-wn9n4\") pod \"must-gather-fwh9z\" (UID: \"7eb6f778-95ea-4aaf-8500-febf6b348523\") " pod="openshift-must-gather-rwtsd/must-gather-fwh9z"
May 26 03:07:30.398336 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.398242    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rwtsd/must-gather-fwh9z"
May 26 03:07:30.522111 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:30.522076    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-rwtsd/must-gather-fwh9z"]
May 26 03:07:30.525105 ip-10-0-139-173 kubenswrapper[2551]: W0526 03:07:30.525068    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7eb6f778_95ea_4aaf_8500_febf6b348523.slice/crio-728e3cd9bdaa5674dedc453a28c3584c471c4942a9ecd36116ba82bd56c999dc WatchSource:0}: Error finding container 728e3cd9bdaa5674dedc453a28c3584c471c4942a9ecd36116ba82bd56c999dc: Status 404 returned error can't find the container with id 728e3cd9bdaa5674dedc453a28c3584c471c4942a9ecd36116ba82bd56c999dc
May 26 03:07:31.182737 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:31.182675    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rwtsd/must-gather-fwh9z" event={"ID":"7eb6f778-95ea-4aaf-8500-febf6b348523","Type":"ContainerStarted","Data":"728e3cd9bdaa5674dedc453a28c3584c471c4942a9ecd36116ba82bd56c999dc"}
May 26 03:07:37.207779 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:37.207403    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rwtsd/must-gather-fwh9z" event={"ID":"7eb6f778-95ea-4aaf-8500-febf6b348523","Type":"ContainerStarted","Data":"c895fbcabbec4724b5a80d16ae0b31ded0452bb317cadc40be9219764e1b79be"}
May 26 03:07:37.207779 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:37.207452    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rwtsd/must-gather-fwh9z" event={"ID":"7eb6f778-95ea-4aaf-8500-febf6b348523","Type":"ContainerStarted","Data":"b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3"}
May 26 03:07:37.221912 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:37.221862    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-rwtsd/must-gather-fwh9z" podStartSLOduration=1.2279249110000001 podStartE2EDuration="7.221847979s" podCreationTimestamp="2026-05-26 03:07:30 +0000 UTC" firstStartedPulling="2026-05-26 03:07:30.527139447 +0000 UTC m=+1888.083928094" lastFinishedPulling="2026-05-26 03:07:36.521062511 +0000 UTC m=+1894.077851162" observedRunningTime="2026-05-26 03:07:37.220568275 +0000 UTC m=+1894.777356944" watchObservedRunningTime="2026-05-26 03:07:37.221847979 +0000 UTC m=+1894.778636648"
May 26 03:07:46.235430 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:46.235392    2551 generic.go:358] "Generic (PLEG): container finished" podID="7eb6f778-95ea-4aaf-8500-febf6b348523" containerID="b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3" exitCode=0
May 26 03:07:46.235870 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:46.235467    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-rwtsd/must-gather-fwh9z" event={"ID":"7eb6f778-95ea-4aaf-8500-febf6b348523","Type":"ContainerDied","Data":"b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3"}
May 26 03:07:46.235870 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:46.235822    2551 scope.go:117] "RemoveContainer" containerID="b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3"
May 26 03:07:47.193752 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:47.193722    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-rwtsd_must-gather-fwh9z_7eb6f778-95ea-4aaf-8500-febf6b348523/gather/0.log"
May 26 03:07:50.145956 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:50.145921    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-kj9cp_995958a8-97ad-4e57-96be-498290f87329/global-pull-secret-syncer/0.log"
May 26 03:07:50.247148 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:50.247112    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-vk2g6_d82f5f80-58cd-4e40-a66e-27301ac76a37/konnectivity-agent/0.log"
May 26 03:07:50.311180 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:50.311147    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-139-173.ec2.internal_8c67b16bff298a6f27ca11c5dc88021f/haproxy/0.log"
May 26 03:07:52.501228 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:52.501192    2551 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-rwtsd/must-gather-fwh9z"]
May 26 03:07:52.501658 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:52.501452    2551 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-must-gather-rwtsd/must-gather-fwh9z" podUID="7eb6f778-95ea-4aaf-8500-febf6b348523" containerName="copy" containerID="cri-o://c895fbcabbec4724b5a80d16ae0b31ded0452bb317cadc40be9219764e1b79be" gracePeriod=2
May 26 03:07:52.506776 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:52.506745    2551 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-rwtsd/must-gather-fwh9z"]
May 26 03:07:52.733265 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:52.733243    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-rwtsd_must-gather-fwh9z_7eb6f778-95ea-4aaf-8500-febf6b348523/copy/0.log"
May 26 03:07:52.733628 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:52.733612    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rwtsd/must-gather-fwh9z"
May 26 03:07:52.736162 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:52.736138    2551 status_manager.go:895] "Failed to get status for pod" podUID="7eb6f778-95ea-4aaf-8500-febf6b348523" pod="openshift-must-gather-rwtsd/must-gather-fwh9z" err="pods \"must-gather-fwh9z\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-rwtsd\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object"
May 26 03:07:52.768605 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:52.768521    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-wn9n4\" (UniqueName: \"kubernetes.io/projected/7eb6f778-95ea-4aaf-8500-febf6b348523-kube-api-access-wn9n4\") pod \"7eb6f778-95ea-4aaf-8500-febf6b348523\" (UID: \"7eb6f778-95ea-4aaf-8500-febf6b348523\") "
May 26 03:07:52.768605 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:52.768567    2551 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7eb6f778-95ea-4aaf-8500-febf6b348523-must-gather-output\") pod \"7eb6f778-95ea-4aaf-8500-febf6b348523\" (UID: \"7eb6f778-95ea-4aaf-8500-febf6b348523\") "
May 26 03:07:52.768959 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:52.768933    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/7eb6f778-95ea-4aaf-8500-febf6b348523-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "7eb6f778-95ea-4aaf-8500-febf6b348523" (UID: "7eb6f778-95ea-4aaf-8500-febf6b348523"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 26 03:07:52.770717 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:52.770693    2551 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/7eb6f778-95ea-4aaf-8500-febf6b348523-kube-api-access-wn9n4" (OuterVolumeSpecName: "kube-api-access-wn9n4") pod "7eb6f778-95ea-4aaf-8500-febf6b348523" (UID: "7eb6f778-95ea-4aaf-8500-febf6b348523"). InnerVolumeSpecName "kube-api-access-wn9n4". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 26 03:07:52.869381 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:52.869338    2551 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-wn9n4\" (UniqueName: \"kubernetes.io/projected/7eb6f778-95ea-4aaf-8500-febf6b348523-kube-api-access-wn9n4\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:52.869381 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:52.869372    2551 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/7eb6f778-95ea-4aaf-8500-febf6b348523-must-gather-output\") on node \"ip-10-0-139-173.ec2.internal\" DevicePath \"\""
May 26 03:07:53.048272 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.048185    2551 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="7eb6f778-95ea-4aaf-8500-febf6b348523" path="/var/lib/kubelet/pods/7eb6f778-95ea-4aaf-8500-febf6b348523/volumes"
May 26 03:07:53.049494 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.049465    2551 status_manager.go:895] "Failed to get status for pod" podUID="7eb6f778-95ea-4aaf-8500-febf6b348523" pod="openshift-must-gather-rwtsd/must-gather-fwh9z" err="pods \"must-gather-fwh9z\" is forbidden: User \"system:node:ip-10-0-139-173.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-rwtsd\": no relationship found between node 'ip-10-0-139-173.ec2.internal' and this object"
May 26 03:07:53.258056 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.258028    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-rwtsd_must-gather-fwh9z_7eb6f778-95ea-4aaf-8500-febf6b348523/copy/0.log"
May 26 03:07:53.258389 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.258364    2551 generic.go:358] "Generic (PLEG): container finished" podID="7eb6f778-95ea-4aaf-8500-febf6b348523" containerID="c895fbcabbec4724b5a80d16ae0b31ded0452bb317cadc40be9219764e1b79be" exitCode=143
May 26 03:07:53.258456 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.258422    2551 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-rwtsd/must-gather-fwh9z"
May 26 03:07:53.258494 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.258464    2551 scope.go:117] "RemoveContainer" containerID="c895fbcabbec4724b5a80d16ae0b31ded0452bb317cadc40be9219764e1b79be"
May 26 03:07:53.265809 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.265791    2551 scope.go:117] "RemoveContainer" containerID="b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3"
May 26 03:07:53.276989 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.276968    2551 scope.go:117] "RemoveContainer" containerID="c895fbcabbec4724b5a80d16ae0b31ded0452bb317cadc40be9219764e1b79be"
May 26 03:07:53.277253 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:07:53.277235    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"c895fbcabbec4724b5a80d16ae0b31ded0452bb317cadc40be9219764e1b79be\": container with ID starting with c895fbcabbec4724b5a80d16ae0b31ded0452bb317cadc40be9219764e1b79be not found: ID does not exist" containerID="c895fbcabbec4724b5a80d16ae0b31ded0452bb317cadc40be9219764e1b79be"
May 26 03:07:53.277304 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.277265    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"c895fbcabbec4724b5a80d16ae0b31ded0452bb317cadc40be9219764e1b79be"} err="failed to get container status \"c895fbcabbec4724b5a80d16ae0b31ded0452bb317cadc40be9219764e1b79be\": rpc error: code = NotFound desc = could not find container \"c895fbcabbec4724b5a80d16ae0b31ded0452bb317cadc40be9219764e1b79be\": container with ID starting with c895fbcabbec4724b5a80d16ae0b31ded0452bb317cadc40be9219764e1b79be not found: ID does not exist"
May 26 03:07:53.277304 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.277285    2551 scope.go:117] "RemoveContainer" containerID="b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3"
May 26 03:07:53.277488 ip-10-0-139-173 kubenswrapper[2551]: E0526 03:07:53.277476    2551 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3\": container with ID starting with b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3 not found: ID does not exist" containerID="b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3"
May 26 03:07:53.277529 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.277491    2551 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3"} err="failed to get container status \"b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3\": rpc error: code = NotFound desc = could not find container \"b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3\": container with ID starting with b745d37109836998315f4504cb96c2f09c613932b646b57e4e6d451b9f61afd3 not found: ID does not exist"
May 26 03:07:53.701396 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.701367    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-z5hcp_1fc63ba7-ff6b-4fad-8d89-07c470d123f1/kube-state-metrics/0.log"
May 26 03:07:53.725355 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.725325    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-z5hcp_1fc63ba7-ff6b-4fad-8d89-07c470d123f1/kube-rbac-proxy-main/0.log"
May 26 03:07:53.749954 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.749931    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-z5hcp_1fc63ba7-ff6b-4fad-8d89-07c470d123f1/kube-rbac-proxy-self/0.log"
May 26 03:07:53.776463 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.776441    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_metrics-server-75bc9899bc-v52lw_9f65a295-2483-43b2-95cb-aa7c79471872/metrics-server/0.log"
May 26 03:07:53.798465 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.798437    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-7b5ld_7c105fe7-9de3-4666-9ea6-c6fc03128dbf/monitoring-plugin/0.log"
May 26 03:07:53.964606 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.964519    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zjmg7_3bdad698-da5b-4759-be30-9011e4728f65/node-exporter/0.log"
May 26 03:07:53.982202 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:53.982175    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zjmg7_3bdad698-da5b-4759-be30-9011e4728f65/kube-rbac-proxy/0.log"
May 26 03:07:54.000518 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:54.000486    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-zjmg7_3bdad698-da5b-4759-be30-9011e4728f65/init-textfile/0.log"
May 26 03:07:54.025793 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:54.025766    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-8vthv_73274278-9bc8-4b7f-9e3c-2ae1b9a05e62/kube-rbac-proxy-main/0.log"
May 26 03:07:54.045956 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:54.045926    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-8vthv_73274278-9bc8-4b7f-9e3c-2ae1b9a05e62/kube-rbac-proxy-self/0.log"
May 26 03:07:54.065520 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:54.065494    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_openshift-state-metrics-9c959559f-8vthv_73274278-9bc8-4b7f-9e3c-2ae1b9a05e62/openshift-state-metrics/0.log"
May 26 03:07:54.320158 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:54.320137    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7bbff8d7c5-2hkhw_0ef0b8fc-02e3-44f3-9b8d-54740392b572/telemeter-client/0.log"
May 26 03:07:54.340290 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:54.340266    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7bbff8d7c5-2hkhw_0ef0b8fc-02e3-44f3-9b8d-54740392b572/reload/0.log"
May 26 03:07:54.357522 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:54.357500    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_telemeter-client-7bbff8d7c5-2hkhw_0ef0b8fc-02e3-44f3-9b8d-54740392b572/kube-rbac-proxy/0.log"
May 26 03:07:56.020051 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.020021    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/1.log"
May 26 03:07:56.025290 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.025267    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-5pgsl_fb87f3ac-7bba-4002-8d34-fadbd525b32a/console-operator/2.log"
May 26 03:07:56.335955 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.335927    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-74f8df89f5-rhbtk_f9d8feaf-5d5b-41c9-a635-f475a317ce13/console/0.log"
May 26 03:07:56.361703 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.361670    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-jst4x_5f7b0578-f539-440b-9f80-b74ac83d5e09/download-server/0.log"
May 26 03:07:56.655489 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.655412    2551 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"]
May 26 03:07:56.655742 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.655716    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eb6f778-95ea-4aaf-8500-febf6b348523" containerName="copy"
May 26 03:07:56.655742 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.655741    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eb6f778-95ea-4aaf-8500-febf6b348523" containerName="copy"
May 26 03:07:56.656126 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.655771    2551 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="7eb6f778-95ea-4aaf-8500-febf6b348523" containerName="gather"
May 26 03:07:56.656126 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.655779    2551 state_mem.go:107] "Deleted CPUSet assignment" podUID="7eb6f778-95ea-4aaf-8500-febf6b348523" containerName="gather"
May 26 03:07:56.656126 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.655846    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="7eb6f778-95ea-4aaf-8500-febf6b348523" containerName="gather"
May 26 03:07:56.656126 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.655869    2551 memory_manager.go:356] "RemoveStaleState removing state" podUID="7eb6f778-95ea-4aaf-8500-febf6b348523" containerName="copy"
May 26 03:07:56.659611 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.659590    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.661660 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.661640    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bftcf\"/\"kube-root-ca.crt\""
May 26 03:07:56.662449 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.662430    2551 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-bftcf\"/\"default-dockercfg-w524c\""
May 26 03:07:56.662449 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.662441    2551 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-bftcf\"/\"openshift-service-ca.crt\""
May 26 03:07:56.665588 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.665566    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"]
May 26 03:07:56.704443 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.704412    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-frtw4\" (UniqueName: \"kubernetes.io/projected/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-kube-api-access-frtw4\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.704612 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.704461    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-podres\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.704612 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.704542    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-proc\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.704612 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.704577    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-lib-modules\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.704713 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.704610    2551 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-sys\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.737604 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.737560    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-storage-operator_volume-data-source-validator-5cf8c89d64-g2gst_974b2dec-e8a5-4ac2-af9d-46e5acdfc7f4/volume-data-source-validator/0.log"
May 26 03:07:56.805263 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.805230    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-proc\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.805263 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.805262    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-lib-modules\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.805463 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.805295    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-sys\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.805463 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.805343    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-frtw4\" (UniqueName: \"kubernetes.io/projected/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-kube-api-access-frtw4\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.805463 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.805357    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-proc\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.805463 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.805386    2551 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-podres\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.805463 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.805433    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-lib-modules\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.805463 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.805431    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-sys\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.805660 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.805480    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-podres\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.812640 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.812613    2551 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-frtw4\" (UniqueName: \"kubernetes.io/projected/e339e0f6-c1c5-420c-892c-f9e9e750bb6a-kube-api-access-frtw4\") pod \"perf-node-gather-daemonset-5gq8d\" (UID: \"e339e0f6-c1c5-420c-892c-f9e9e750bb6a\") " pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:56.970607 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:56.970514    2551 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:57.097824 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:57.097789    2551 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"]
May 26 03:07:57.101188 ip-10-0-139-173 kubenswrapper[2551]: W0526 03:07:57.101159    2551 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-pode339e0f6_c1c5_420c_892c_f9e9e750bb6a.slice/crio-cd66e4b956a6884316c8bc0ea19d020bfec61e2bc46e06382204beb15df66c84 WatchSource:0}: Error finding container cd66e4b956a6884316c8bc0ea19d020bfec61e2bc46e06382204beb15df66c84: Status 404 returned error can't find the container with id cd66e4b956a6884316c8bc0ea19d020bfec61e2bc46e06382204beb15df66c84
May 26 03:07:57.273509 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:57.273470    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d" event={"ID":"e339e0f6-c1c5-420c-892c-f9e9e750bb6a","Type":"ContainerStarted","Data":"0974e18d701c8c83dd11c2eef7dab18b70c6894cfdc9e4180dcccd9caee3763c"}
May 26 03:07:57.273509 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:57.273509    2551 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d" event={"ID":"e339e0f6-c1c5-420c-892c-f9e9e750bb6a","Type":"ContainerStarted","Data":"cd66e4b956a6884316c8bc0ea19d020bfec61e2bc46e06382204beb15df66c84"}
May 26 03:07:57.273732 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:57.273539    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:07:57.288693 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:57.288643    2551 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d" podStartSLOduration=1.2886281689999999 podStartE2EDuration="1.288628169s" podCreationTimestamp="2026-05-26 03:07:56 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-26 03:07:57.28667309 +0000 UTC m=+1914.843461763" watchObservedRunningTime="2026-05-26 03:07:57.288628169 +0000 UTC m=+1914.845416837"
May 26 03:07:57.323782 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:57.323754    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r4986_a7e61c19-3818-47b3-82b7-bc5b51bb09de/dns/0.log"
May 26 03:07:57.341471 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:57.341450    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-r4986_a7e61c19-3818-47b3-82b7-bc5b51bb09de/kube-rbac-proxy/0.log"
May 26 03:07:57.420375 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:57.420341    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-j55gs_7cef8ed3-f979-4a27-983b-8f89cfa6b7ae/dns-node-resolver/0.log"
May 26 03:07:57.789348 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:57.789319    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_image-registry-c6ccbcd49-fwcnf_3895dbfa-efcf-4d00-a0ee-fec2b10af485/registry/0.log"
May 26 03:07:57.853403 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:57.853375    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-q8dr4_7dd95e00-cc75-4517-a4cb-2484530f0fbd/node-ca/0.log"
May 26 03:07:58.803141 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:58.803104    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-tpfbl_96779027-bcda-438b-924c-3441dd3c4b9d/serve-healthcheck-canary/0.log"
May 26 03:07:59.106758 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:59.106671    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-p9x4q_1511080f-2218-4a2b-8395-1ae813c119b2/insights-operator/1.log"
May 26 03:07:59.106758 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:59.106668    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-operator-78bdfb888c-p9x4q_1511080f-2218-4a2b-8395-1ae813c119b2/insights-operator/0.log"
May 26 03:07:59.178222 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:59.178194    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5vxw6_306321fb-101c-4452-a4db-fc8e753b5d53/kube-rbac-proxy/0.log"
May 26 03:07:59.194820 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:59.194789    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5vxw6_306321fb-101c-4452-a4db-fc8e753b5d53/exporter/0.log"
May 26 03:07:59.212202 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:07:59.212177    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-5vxw6_306321fb-101c-4452-a4db-fc8e753b5d53/extractor/0.log"
May 26 03:08:03.219522 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:03.219472    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-xs5pj_355b4827-cbde-4c11-813c-08ed3b73ad7d/kube-storage-version-migrator-operator/1.log"
May 26 03:08:03.220441 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:03.220420    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-xs5pj_355b4827-cbde-4c11-813c-08ed3b73ad7d/kube-storage-version-migrator-operator/0.log"
May 26 03:08:03.287121 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:03.287092    2551 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-bftcf/perf-node-gather-daemonset-5gq8d"
May 26 03:08:04.059800 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:04.059770    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5x5dt_5e26384c-7c3d-4dad-bab9-a74ec1259042/kube-multus-additional-cni-plugins/0.log"
May 26 03:08:04.077849 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:04.077822    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5x5dt_5e26384c-7c3d-4dad-bab9-a74ec1259042/egress-router-binary-copy/0.log"
May 26 03:08:04.094779 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:04.094751    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5x5dt_5e26384c-7c3d-4dad-bab9-a74ec1259042/cni-plugins/0.log"
May 26 03:08:04.111807 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:04.111779    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5x5dt_5e26384c-7c3d-4dad-bab9-a74ec1259042/bond-cni-plugin/0.log"
May 26 03:08:04.129018 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:04.128992    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5x5dt_5e26384c-7c3d-4dad-bab9-a74ec1259042/routeoverride-cni/0.log"
May 26 03:08:04.146157 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:04.146125    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5x5dt_5e26384c-7c3d-4dad-bab9-a74ec1259042/whereabouts-cni-bincopy/0.log"
May 26 03:08:04.163200 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:04.163181    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-5x5dt_5e26384c-7c3d-4dad-bab9-a74ec1259042/whereabouts-cni/0.log"
May 26 03:08:04.452543 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:04.452461    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-gmx7w_8df2d22d-804e-42c6-9100-d32e0f707aae/kube-multus/0.log"
May 26 03:08:04.469315 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:04.469283    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-29jmf_9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8/network-metrics-daemon/0.log"
May 26 03:08:04.485836 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:04.485809    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-29jmf_9da4a71a-9ddf-4352-9fc7-6cd9f46f35b8/kube-rbac-proxy/0.log"
May 26 03:08:05.821523 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:05.821494    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-controller/0.log"
May 26 03:08:05.838651 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:05.838617    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/0.log"
May 26 03:08:05.846863 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:05.846827    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovn-acl-logging/1.log"
May 26 03:08:05.865702 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:05.865677    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/kube-rbac-proxy-node/0.log"
May 26 03:08:05.886836 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:05.886802    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/kube-rbac-proxy-ovn-metrics/0.log"
May 26 03:08:05.910930 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:05.910886    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/northd/0.log"
May 26 03:08:05.929073 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:05.929047    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/nbdb/0.log"
May 26 03:08:05.949101 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:05.949069    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/sbdb/0.log"
May 26 03:08:06.039939 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:06.039887    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-t8q5v_ac847df6-96a0-478b-8c22-796d6809971a/ovnkube-controller/0.log"
May 26 03:08:07.010637 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:07.010606    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-wzhws_53dbc3bb-daa3-4d5c-a004-f5046a91b1ae/network-check-target-container/0.log"
May 26 03:08:07.892227 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:07.892199    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-4w92x_5ffa480f-baaf-4a18-adf0-dd6ed1d2915a/iptables-alerter/0.log"
May 26 03:08:08.499144 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:08.499116    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-sq9nq_c578cb33-6d42-4ae0-a636-58b4be832e7d/tuned/0.log"
May 26 03:08:10.864881 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:10.864852    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-57b5b68b8-wptxx_49675dc6-0f22-46e0-a1cd-5408f5598780/service-ca-operator/1.log"
May 26 03:08:10.865740 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:10.865723    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca-operator_service-ca-operator-57b5b68b8-wptxx_49675dc6-0f22-46e0-a1cd-5408f5598780/service-ca-operator/0.log"
May 26 03:08:11.140574 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:11.140491    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-service-ca_service-ca-67d667c7d-67ntt_30e7f33a-343c-4873-910d-4596b35b3484/service-ca-controller/0.log"
May 26 03:08:11.539890 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:11.539816    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-md5xq_f2d576e5-b71f-438e-8dde-aebb68efe86e/csi-driver/0.log"
May 26 03:08:11.559560 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:11.559537    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-md5xq_f2d576e5-b71f-438e-8dde-aebb68efe86e/csi-node-driver-registrar/0.log"
May 26 03:08:11.577205 ip-10-0-139-173 kubenswrapper[2551]: I0526 03:08:11.577190    2551 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-csi-drivers_aws-ebs-csi-driver-node-md5xq_f2d576e5-b71f-438e-8dde-aebb68efe86e/csi-liveness-probe/0.log"