May 28 08:09:25.943221 ip-10-0-137-158 systemd[1]: kubelet.service: Failed to load environment files: No such file or directory
May 28 08:09:25.943234 ip-10-0-137-158 systemd[1]: kubelet.service: Failed to run 'start-pre' task: No such file or directory
May 28 08:09:25.943243 ip-10-0-137-158 systemd[1]: kubelet.service: Failed with result 'resources'.
May 28 08:09:25.943562 ip-10-0-137-158 systemd[1]: Failed to start Kubernetes Kubelet.
May 28 08:09:36.164095 ip-10-0-137-158 systemd[1]: kubelet.service: Failed to schedule restart job: Unit crio.service not found.
May 28 08:09:36.164114 ip-10-0-137-158 systemd[1]: kubelet.service: Failed with result 'resources'.
-- Boot e63fc74b75154f03aef371b0218f5321 --
May 28 08:12:01.429968 ip-10-0-137-158 systemd[1]: Starting Kubernetes Kubelet...
May 28 08:12:01.939324 ip-10-0-137-158 kubenswrapper[2577]: Flag --container-runtime-endpoint has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 08:12:01.939324 ip-10-0-137-158 kubenswrapper[2577]: Flag --minimum-container-ttl-duration has been deprecated, Use --eviction-hard or --eviction-soft instead. Will be removed in a future version.
May 28 08:12:01.939324 ip-10-0-137-158 kubenswrapper[2577]: Flag --volume-plugin-dir has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 08:12:01.939324 ip-10-0-137-158 kubenswrapper[2577]: Flag --pod-infra-container-image has been deprecated, will be removed in 1.35. Image garbage collector will get sandbox image information from CRI.
May 28 08:12:01.939324 ip-10-0-137-158 kubenswrapper[2577]: Flag --system-reserved has been deprecated, This parameter should be set via the config file specified by the Kubelet's --config flag. See https://kubernetes.io/docs/tasks/administer-cluster/kubelet-config-file/ for more information.
May 28 08:12:01.940898 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.940798    2577 server.go:212] "--pod-infra-container-image will not be pruned by the image garbage collector in kubelet and should also be set in the remote runtime"
May 28 08:12:01.944906 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944885    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 08:12:01.944906 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944905    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944909    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944912    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944915    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944920    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944923    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944926    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944929    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944932    2577 feature_gate.go:328] unrecognized feature gate: Example
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944934    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944937    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944940    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944950    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944953    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944956    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944959    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944963    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944967    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944971    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944974    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 08:12:01.944980 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944977    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944979    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944982    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944985    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944987    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944990    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944993    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944995    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.944998    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945001    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945003    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945006    2577 feature_gate.go:328] unrecognized feature gate: Example2
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945010    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945013    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945016    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945019    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945022    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945024    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945026    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 08:12:01.945456 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945031    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945034    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945036    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945038    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945041    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945043    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945046    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945048    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945051    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945053    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945056    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945058    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945061    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945063    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945066    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945069    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945072    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945075    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945077    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945080    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 08:12:01.945946 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945082    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945086    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945088    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945091    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945093    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945096    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945098    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945100    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945104    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945107    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945110    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945112    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945115    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945117    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945120    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945122    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945124    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945127    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945129    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945133    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 08:12:01.946429 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945136    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945138    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945141    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945143    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945146    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945149    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945545    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945550    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945553    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945556    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945559    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945563    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945565    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945568    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945570    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945573    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945575    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945578    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945580    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945583    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 08:12:01.946969 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945586    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945589    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945592    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945594    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945597    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945599    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945601    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945627    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945631    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945635    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945639    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945642    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945646    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945648    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945667    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945671    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945673    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945676    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945679    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 08:12:01.947462 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945682    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945685    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945687    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945690    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945692    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945695    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945698    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945700    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945703    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945706    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945708    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945712    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945714    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945717    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945720    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945723    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945725    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945728    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945730    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945733    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 08:12:01.947951 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945735    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945738    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945740    2577 feature_gate.go:328] unrecognized feature gate: Example
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945743    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945746    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945749    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945752    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945754    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945757    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945759    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945761    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945765    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945769    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945772    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945774    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945776    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945779    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945781    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945784    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945786    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 08:12:01.948437 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945788    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945791    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945793    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945796    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945798    2577 feature_gate.go:328] unrecognized feature gate: Example2
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945801    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945803    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945806    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945809    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945811    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945814    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945816    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.945819    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.947308    2577 flags.go:64] FLAG: --address="0.0.0.0"
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.947320    2577 flags.go:64] FLAG: --allowed-unsafe-sysctls="[]"
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948160    2577 flags.go:64] FLAG: --anonymous-auth="true"
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948166    2577 flags.go:64] FLAG: --application-metrics-count-limit="100"
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948172    2577 flags.go:64] FLAG: --authentication-token-webhook="false"
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948175    2577 flags.go:64] FLAG: --authentication-token-webhook-cache-ttl="2m0s"
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948180    2577 flags.go:64] FLAG: --authorization-mode="AlwaysAllow"
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948184    2577 flags.go:64] FLAG: --authorization-webhook-cache-authorized-ttl="5m0s"
May 28 08:12:01.948956 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948188    2577 flags.go:64] FLAG: --authorization-webhook-cache-unauthorized-ttl="30s"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948191    2577 flags.go:64] FLAG: --boot-id-file="/proc/sys/kernel/random/boot_id"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948195    2577 flags.go:64] FLAG: --bootstrap-kubeconfig="/etc/kubernetes/kubeconfig"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948199    2577 flags.go:64] FLAG: --cert-dir="/var/lib/kubelet/pki"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948202    2577 flags.go:64] FLAG: --cgroup-driver="cgroupfs"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948205    2577 flags.go:64] FLAG: --cgroup-root=""
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948208    2577 flags.go:64] FLAG: --cgroups-per-qos="true"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948211    2577 flags.go:64] FLAG: --client-ca-file=""
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948214    2577 flags.go:64] FLAG: --cloud-config=""
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948216    2577 flags.go:64] FLAG: --cloud-provider="external"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948219    2577 flags.go:64] FLAG: --cluster-dns="[]"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948225    2577 flags.go:64] FLAG: --cluster-domain=""
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948228    2577 flags.go:64] FLAG: --config="/etc/kubernetes/kubelet.conf"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948231    2577 flags.go:64] FLAG: --config-dir=""
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948234    2577 flags.go:64] FLAG: --container-hints="/etc/cadvisor/container_hints.json"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948238    2577 flags.go:64] FLAG: --container-log-max-files="5"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948242    2577 flags.go:64] FLAG: --container-log-max-size="10Mi"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948245    2577 flags.go:64] FLAG: --container-runtime-endpoint="/var/run/crio/crio.sock"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948248    2577 flags.go:64] FLAG: --containerd="/run/containerd/containerd.sock"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948253    2577 flags.go:64] FLAG: --containerd-namespace="k8s.io"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948256    2577 flags.go:64] FLAG: --contention-profiling="false"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948259    2577 flags.go:64] FLAG: --cpu-cfs-quota="true"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948262    2577 flags.go:64] FLAG: --cpu-cfs-quota-period="100ms"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948265    2577 flags.go:64] FLAG: --cpu-manager-policy="none"
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948268    2577 flags.go:64] FLAG: --cpu-manager-policy-options=""
May 28 08:12:01.949477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948272    2577 flags.go:64] FLAG: --cpu-manager-reconcile-period="10s"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948275    2577 flags.go:64] FLAG: --enable-controller-attach-detach="true"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948279    2577 flags.go:64] FLAG: --enable-debugging-handlers="true"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948282    2577 flags.go:64] FLAG: --enable-load-reader="false"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948285    2577 flags.go:64] FLAG: --enable-server="true"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948288    2577 flags.go:64] FLAG: --enforce-node-allocatable="[pods]"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948293    2577 flags.go:64] FLAG: --event-burst="100"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948296    2577 flags.go:64] FLAG: --event-qps="50"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948300    2577 flags.go:64] FLAG: --event-storage-age-limit="default=0"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948303    2577 flags.go:64] FLAG: --event-storage-event-limit="default=0"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948307    2577 flags.go:64] FLAG: --eviction-hard=""
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948310    2577 flags.go:64] FLAG: --eviction-max-pod-grace-period="0"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948314    2577 flags.go:64] FLAG: --eviction-minimum-reclaim=""
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948317    2577 flags.go:64] FLAG: --eviction-pressure-transition-period="5m0s"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948320    2577 flags.go:64] FLAG: --eviction-soft=""
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948323    2577 flags.go:64] FLAG: --eviction-soft-grace-period=""
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948326    2577 flags.go:64] FLAG: --exit-on-lock-contention="false"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948329    2577 flags.go:64] FLAG: --experimental-allocatable-ignore-eviction="false"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948332    2577 flags.go:64] FLAG: --experimental-mounter-path=""
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948335    2577 flags.go:64] FLAG: --fail-cgroupv1="false"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948337    2577 flags.go:64] FLAG: --fail-swap-on="true"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948340    2577 flags.go:64] FLAG: --feature-gates=""
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948344    2577 flags.go:64] FLAG: --file-check-frequency="20s"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948347    2577 flags.go:64] FLAG: --global-housekeeping-interval="1m0s"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948350    2577 flags.go:64] FLAG: --hairpin-mode="promiscuous-bridge"
May 28 08:12:01.950181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948353    2577 flags.go:64] FLAG: --healthz-bind-address="127.0.0.1"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948357    2577 flags.go:64] FLAG: --healthz-port="10248"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948361    2577 flags.go:64] FLAG: --help="false"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948364    2577 flags.go:64] FLAG: --hostname-override="ip-10-0-137-158.ec2.internal"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948367    2577 flags.go:64] FLAG: --housekeeping-interval="10s"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948370    2577 flags.go:64] FLAG: --http-check-frequency="20s"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948373    2577 flags.go:64] FLAG: --image-credential-provider-bin-dir="/usr/libexec/kubelet-image-credential-provider-plugins"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948376    2577 flags.go:64] FLAG: --image-credential-provider-config="/etc/kubernetes/credential-providers/ecr-credential-provider.yaml"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948379    2577 flags.go:64] FLAG: --image-gc-high-threshold="85"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948382    2577 flags.go:64] FLAG: --image-gc-low-threshold="80"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948385    2577 flags.go:64] FLAG: --image-service-endpoint=""
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948388    2577 flags.go:64] FLAG: --kernel-memcg-notification="false"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948391    2577 flags.go:64] FLAG: --kube-api-burst="100"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948394    2577 flags.go:64] FLAG: --kube-api-content-type="application/vnd.kubernetes.protobuf"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948397    2577 flags.go:64] FLAG: --kube-api-qps="50"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948400    2577 flags.go:64] FLAG: --kube-reserved=""
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948403    2577 flags.go:64] FLAG: --kube-reserved-cgroup=""
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948406    2577 flags.go:64] FLAG: --kubeconfig="/var/lib/kubelet/kubeconfig"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948409    2577 flags.go:64] FLAG: --kubelet-cgroups=""
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948412    2577 flags.go:64] FLAG: --local-storage-capacity-isolation="true"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948415    2577 flags.go:64] FLAG: --lock-file=""
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948417    2577 flags.go:64] FLAG: --log-cadvisor-usage="false"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948420    2577 flags.go:64] FLAG: --log-flush-frequency="5s"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948423    2577 flags.go:64] FLAG: --log-json-info-buffer-size="0"
May 28 08:12:01.950827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948429    2577 flags.go:64] FLAG: --log-json-split-stream="false"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948432    2577 flags.go:64] FLAG: --log-text-info-buffer-size="0"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948435    2577 flags.go:64] FLAG: --log-text-split-stream="false"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948438    2577 flags.go:64] FLAG: --logging-format="text"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948441    2577 flags.go:64] FLAG: --machine-id-file="/etc/machine-id,/var/lib/dbus/machine-id"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948444    2577 flags.go:64] FLAG: --make-iptables-util-chains="true"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948447    2577 flags.go:64] FLAG: --manifest-url=""
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948450    2577 flags.go:64] FLAG: --manifest-url-header=""
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948455    2577 flags.go:64] FLAG: --max-housekeeping-interval="15s"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948459    2577 flags.go:64] FLAG: --max-open-files="1000000"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948463    2577 flags.go:64] FLAG: --max-pods="110"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948467    2577 flags.go:64] FLAG: --maximum-dead-containers="-1"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948470    2577 flags.go:64] FLAG: --maximum-dead-containers-per-container="1"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948473    2577 flags.go:64] FLAG: --memory-manager-policy="None"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948476    2577 flags.go:64] FLAG: --minimum-container-ttl-duration="6m0s"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948479    2577 flags.go:64] FLAG: --minimum-image-ttl-duration="2m0s"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948482    2577 flags.go:64] FLAG: --node-ip="0.0.0.0"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948485    2577 flags.go:64] FLAG: --node-labels="node-role.kubernetes.io/worker=,node.openshift.io/os_id=rhel"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948495    2577 flags.go:64] FLAG: --node-status-max-images="50"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948498    2577 flags.go:64] FLAG: --node-status-update-frequency="10s"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948501    2577 flags.go:64] FLAG: --oom-score-adj="-999"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948504    2577 flags.go:64] FLAG: --pod-cidr=""
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948507    2577 flags.go:64] FLAG: --pod-infra-container-image="quay.io/openshift-release-dev/ocp-v4.0-art-dev@sha256:afe88671bf163c42b467d159dc6f0b784dbe28b1ed44e171edc9c06fd99cc6fc"
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948513    2577 flags.go:64] FLAG: --pod-manifest-path=""
May 28 08:12:01.951429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948516    2577 flags.go:64] FLAG: --pod-max-pids="-1"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948519    2577 flags.go:64] FLAG: --pods-per-core="0"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948522    2577 flags.go:64] FLAG: --port="10250"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948525    2577 flags.go:64] FLAG: --protect-kernel-defaults="false"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948529    2577 flags.go:64] FLAG: --provider-id="aws:///us-east-1a/i-0f49e576ae240963f"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948532    2577 flags.go:64] FLAG: --qos-reserved=""
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948535    2577 flags.go:64] FLAG: --read-only-port="10255"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948538    2577 flags.go:64] FLAG: --register-node="true"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948541    2577 flags.go:64] FLAG: --register-schedulable="true"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948544    2577 flags.go:64] FLAG: --register-with-taints=""
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948552    2577 flags.go:64] FLAG: --registry-burst="10"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948555    2577 flags.go:64] FLAG: --registry-qps="5"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948558    2577 flags.go:64] FLAG: --reserved-cpus=""
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948560    2577 flags.go:64] FLAG: --reserved-memory=""
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948564    2577 flags.go:64] FLAG: --resolv-conf="/etc/resolv.conf"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948567    2577 flags.go:64] FLAG: --root-dir="/var/lib/kubelet"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948570    2577 flags.go:64] FLAG: --rotate-certificates="false"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948573    2577 flags.go:64] FLAG: --rotate-server-certificates="false"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948576    2577 flags.go:64] FLAG: --runonce="false"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948579    2577 flags.go:64] FLAG: --runtime-cgroups="/system.slice/crio.service"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948583    2577 flags.go:64] FLAG: --runtime-request-timeout="2m0s"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948586    2577 flags.go:64] FLAG: --seccomp-default="false"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948589    2577 flags.go:64] FLAG: --serialize-image-pulls="true"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948591    2577 flags.go:64] FLAG: --storage-driver-buffer-duration="1m0s"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948594    2577 flags.go:64] FLAG: --storage-driver-db="cadvisor"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948597    2577 flags.go:64] FLAG: --storage-driver-host="localhost:8086"
May 28 08:12:01.952055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948600    2577 flags.go:64] FLAG: --storage-driver-password="root"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948603    2577 flags.go:64] FLAG: --storage-driver-secure="false"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948606    2577 flags.go:64] FLAG: --storage-driver-table="stats"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948609    2577 flags.go:64] FLAG: --storage-driver-user="root"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948612    2577 flags.go:64] FLAG: --streaming-connection-idle-timeout="4h0m0s"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948615    2577 flags.go:64] FLAG: --sync-frequency="1m0s"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948618    2577 flags.go:64] FLAG: --system-cgroups=""
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948621    2577 flags.go:64] FLAG: --system-reserved="cpu=500m,ephemeral-storage=1Gi,memory=1Gi"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948626    2577 flags.go:64] FLAG: --system-reserved-cgroup=""
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948629    2577 flags.go:64] FLAG: --tls-cert-file=""
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948632    2577 flags.go:64] FLAG: --tls-cipher-suites="[]"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948636    2577 flags.go:64] FLAG: --tls-min-version=""
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948639    2577 flags.go:64] FLAG: --tls-private-key-file=""
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948642    2577 flags.go:64] FLAG: --topology-manager-policy="none"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948645    2577 flags.go:64] FLAG: --topology-manager-policy-options=""
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948648    2577 flags.go:64] FLAG: --topology-manager-scope="container"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948666    2577 flags.go:64] FLAG: --v="2"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948672    2577 flags.go:64] FLAG: --version="false"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948676    2577 flags.go:64] FLAG: --vmodule=""
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948681    2577 flags.go:64] FLAG: --volume-plugin-dir="/etc/kubernetes/kubelet-plugins/volume/exec"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.948684    2577 flags.go:64] FLAG: --volume-stats-agg-period="1m0s"
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948790    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948794    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948801    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 08:12:01.952690 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948805    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948808    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948811    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948814    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948817    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948820    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948822    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948825    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948827    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948830    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948832    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948835    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948839    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948842    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948845    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948848    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948852    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948854    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948857    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 08:12:01.953257 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948860    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948863    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948865    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948868    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948871    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948873    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948875    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948878    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948881    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948883    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948885    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948889    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948894    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948898    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948900    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948903    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948905    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948908    2577 feature_gate.go:328] unrecognized feature gate: Example
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948911    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948913    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 08:12:01.953759 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948916    2577 feature_gate.go:328] unrecognized feature gate: Example2
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948918    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948921    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948923    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948926    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948928    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948931    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948933    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948936    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948939    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948942    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948944    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948947    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948949    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948952    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948954    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948957    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948959    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948962    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948964    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 08:12:01.954249 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948967    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948970    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948972    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948975    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948979    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948981    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948984    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948986    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948989    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948991    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948994    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948997    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.948999    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.949002    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.949005    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.949007    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.949010    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.949012    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.949015    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.949017    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 08:12:01.954798 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.949023    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 08:12:01.955317 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.949025    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 08:12:01.955317 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.949028    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 08:12:01.955317 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.949030    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 08:12:01.955317 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.949036    2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 08:12:01.956168 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.956147    2577 server.go:530] "Kubelet version" kubeletVersion="v1.33.11"
May 28 08:12:01.956208 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.956170    2577 server.go:532] "Golang settings" GOGC="" GOMAXPROCS="" GOTRACEBACK=""
May 28 08:12:01.956238 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956220    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 08:12:01.956238 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956225    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 08:12:01.956238 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956229    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 08:12:01.956238 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956232    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 08:12:01.956238 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956235    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 08:12:01.956238 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956238    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 08:12:01.956238 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956241    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956244    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956248    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956250    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956253    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956256    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956258    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956261    2577 feature_gate.go:328] unrecognized feature gate: Example
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956264    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956267    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956270    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956273    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956275    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956278    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956280    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956283    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956286    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956289    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956291    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956294    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 08:12:01.956419 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956296    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956298    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956301    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956303    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956306    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956308    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956311    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956313    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956316    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956318    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956322    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956326    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956329    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956332    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956335    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956337    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956340    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956343    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956346    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 08:12:01.957145 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956348    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956351    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956353    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956356    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956359    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956362    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956364    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956366    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956369    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956371    2577 feature_gate.go:328] unrecognized feature gate: Example2
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956374    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956376    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956379    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956381    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956384    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956387    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956390    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956392    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956394    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956397    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 08:12:01.957816 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956399    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956402    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956404    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956406    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956409    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956411    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956414    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956417    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956419    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956422    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956425    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956427    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956430    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956433    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956435    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956437    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956491    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956548    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956553    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 08:12:01.958314 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956557    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.956564    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.956575    2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957001    2577 feature_gate.go:328] unrecognized feature gate: VSphereConfigurableMaxAllowedBlockVolumesPerNode
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957023    2577 feature_gate.go:328] unrecognized feature gate: InsightsOnDemandDataGather
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957029    2577 feature_gate.go:328] unrecognized feature gate: NutanixMultiSubnets
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957033    2577 feature_gate.go:328] unrecognized feature gate: PreconfiguredUDNAddresses
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957038    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerificationPKI
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957043    2577 feature_gate.go:328] unrecognized feature gate: VolumeGroupSnapshot
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957048    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPIController
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957051    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNS
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957056    2577 feature_gate.go:328] unrecognized feature gate: PinnedImages
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957060    2577 feature_gate.go:328] unrecognized feature gate: NewOLMPreflightPermissionChecks
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957064    2577 feature_gate.go:328] unrecognized feature gate: ExternalSnapshotMetadata
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957068    2577 feature_gate.go:328] unrecognized feature gate: BootcNodeManagement
May 28 08:12:01.958795 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957072    2577 feature_gate.go:328] unrecognized feature gate: ClusterMonitoringConfig
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957076    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesvSphere
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957084    2577 feature_gate.go:328] unrecognized feature gate: NoRegistryClusterOperations
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957088    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNS
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957093    2577 feature_gate.go:328] unrecognized feature gate: EtcdBackendQuota
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957097    2577 feature_gate.go:328] unrecognized feature gate: DNSNameResolver
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957101    2577 feature_gate.go:328] unrecognized feature gate: RouteAdvertisements
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957105    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfig
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957110    2577 feature_gate.go:328] unrecognized feature gate: AWSClusterHostedDNSInstall
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957113    2577 feature_gate.go:328] unrecognized feature gate: ConsolePluginContentSecurityPolicy
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957118    2577 feature_gate.go:328] unrecognized feature gate: MixedCPUsAllocation
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957121    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiDisk
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957125    2577 feature_gate.go:328] unrecognized feature gate: BuildCSIVolumes
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957129    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpoints
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957137    2577 feature_gate.go:328] unrecognized feature gate: MultiDiskSetup
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957141    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIMigration
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957146    2577 feature_gate.go:328] unrecognized feature gate: ImageModeStatusReporting
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957150    2577 feature_gate.go:328] unrecognized feature gate: NewOLMOwnSingleNamespace
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957154    2577 feature_gate.go:328] unrecognized feature gate: NewOLMWebhookProviderOpenshiftServiceCA
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957158    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDC
May 28 08:12:01.959188 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957162    2577 feature_gate.go:328] unrecognized feature gate: NetworkDiagnosticsConfig
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957169    2577 feature_gate.go:328] unrecognized feature gate: Example2
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957173    2577 feature_gate.go:328] unrecognized feature gate: MachineConfigNodes
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957177    2577 feature_gate.go:328] unrecognized feature gate: AlibabaPlatform
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957181    2577 feature_gate.go:328] unrecognized feature gate: AdminNetworkPolicy
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957185    2577 feature_gate.go:328] unrecognized feature gate: GCPClusterHostedDNSInstall
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957190    2577 feature_gate.go:328] unrecognized feature gate: IrreconcilableMachineConfig
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957198    2577 feature_gate.go:328] unrecognized feature gate: NewOLMCatalogdAPIV1Metas
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957203    2577 feature_gate.go:328] unrecognized feature gate: AWSDedicatedHosts
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957207    2577 feature_gate.go:328] unrecognized feature gate: GatewayAPI
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957211    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstall
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957215    2577 feature_gate.go:328] unrecognized feature gate: MultiArchInstallAzure
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957220    2577 feature_gate.go:328] unrecognized feature gate: CPMSMachineNamePrefix
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957224    2577 feature_gate.go:328] unrecognized feature gate: ImageStreamImportMode
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957228    2577 feature_gate.go:328] unrecognized feature gate: AWSServiceLBNetworkSecurityGroup
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957232    2577 feature_gate.go:328] unrecognized feature gate: NetworkLiveMigration
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957237    2577 feature_gate.go:328] unrecognized feature gate: Example
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957241    2577 feature_gate.go:328] unrecognized feature gate: AzureClusterHostedDNSInstall
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957245    2577 feature_gate.go:328] unrecognized feature gate: NetworkSegmentation
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957254    2577 feature_gate.go:328] unrecognized feature gate: AzureDedicatedHosts
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957258    2577 feature_gate.go:328] unrecognized feature gate: UpgradeStatus
May 28 08:12:01.959709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957262    2577 feature_gate.go:328] unrecognized feature gate: AzureWorkloadIdentity
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957266    2577 feature_gate.go:328] unrecognized feature gate: AutomatedEtcdBackup
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957270    2577 feature_gate.go:328] unrecognized feature gate: OpenShiftPodSecurityAdmission
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957274    2577 feature_gate.go:328] unrecognized feature gate: ClusterAPIInstallIBMCloud
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957279    2577 feature_gate.go:328] unrecognized feature gate: KMSEncryptionProvider
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957283    2577 feature_gate.go:328] unrecognized feature gate: BootImageSkewEnforcement
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957287    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerLBSubnetsAWS
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957294    2577 feature_gate.go:351] Setting GA feature gate ServiceAccountTokenNodeBinding=true. It will be removed in a future release.
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957302    2577 feature_gate.go:349] Setting deprecated feature gate KMSv1=true. It will be removed in a future release.
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957309    2577 feature_gate.go:328] unrecognized feature gate: HighlyAvailableArbiter
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957319    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAzure
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957324    2577 feature_gate.go:328] unrecognized feature gate: ClusterVersionOperatorConfiguration
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957329    2577 feature_gate.go:328] unrecognized feature gate: DualReplica
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957335    2577 feature_gate.go:328] unrecognized feature gate: SigstoreImageVerification
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957339    2577 feature_gate.go:328] unrecognized feature gate: OVNObservability
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957343    2577 feature_gate.go:328] unrecognized feature gate: ShortCertRotation
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957348    2577 feature_gate.go:328] unrecognized feature gate: MachineAPIOperatorDisableMachineHealthCheckController
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957352    2577 feature_gate.go:328] unrecognized feature gate: IngressControllerDynamicConfigurationManager
May 28 08:12:01.960229 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957356    2577 feature_gate.go:328] unrecognized feature gate: ExternalOIDCWithUIDAndExtraClaimMappings
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957361    2577 feature_gate.go:328] unrecognized feature gate: GCPCustomAPIEndpointsInstall
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957365    2577 feature_gate.go:328] unrecognized feature gate: InsightsConfigAPI
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957370    2577 feature_gate.go:328] unrecognized feature gate: SignatureStores
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957373    2577 feature_gate.go:328] unrecognized feature gate: NewOLM
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957382    2577 feature_gate.go:328] unrecognized feature gate: VSphereHostVMGroupZonal
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957386    2577 feature_gate.go:328] unrecognized feature gate: MetricsCollectionProfiles
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957390    2577 feature_gate.go:328] unrecognized feature gate: VSphereMixedNodeEnv
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957394    2577 feature_gate.go:328] unrecognized feature gate: AdditionalRoutingCapabilities
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957398    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImagesAWS
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957402    2577 feature_gate.go:328] unrecognized feature gate: SetEIPForNLBIngressController
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957406    2577 feature_gate.go:328] unrecognized feature gate: ManagedBootImages
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957410    2577 feature_gate.go:328] unrecognized feature gate: AzureMultiDisk
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957414    2577 feature_gate.go:328] unrecognized feature gate: DyanmicServiceEndpointIBMCloud
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:01.957419    2577 feature_gate.go:328] unrecognized feature gate: VSphereMultiNetworks
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.957427    2577 feature_gate.go:384] feature gates: {map[DynamicResourceAllocation:false EventedPLEG:false ImageVolume:true KMSv1:true MaxUnavailableStatefulSet:false MinimumKubeletVersion:false MutatingAdmissionPolicy:false NodeSwap:false ProcMountType:true RouteExternalCertificate:true SELinuxMount:false ServiceAccountTokenNodeBinding:true StoragePerformantSecurityPolicy:true TranslateStreamCloseWebsocketRequests:false UserNamespacesPodSecurityStandards:true UserNamespacesSupport:true VolumeAttributesClass:false]}
May 28 08:12:01.960782 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.958585    2577 server.go:962] "Client rotation is on, will bootstrap in background"
May 28 08:12:01.961199 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.960912    2577 bootstrap.go:101] "Use the bootstrap credentials to request a cert, and set kubeconfig to point to the certificate dir"
May 28 08:12:01.962018 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.962007    2577 server.go:1019] "Starting client certificate rotation"
May 28 08:12:01.962128 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.962112    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 08:12:01.962980 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.962969    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kube-apiserver-client-kubelet"
May 28 08:12:01.991273 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.991249    2577 dynamic_cafile_content.go:123] "Loaded a new CA Bundle and Verifier" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 08:12:01.993165 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:01.993138    2577 dynamic_cafile_content.go:161] "Starting controller" name="client-ca-bundle::/etc/kubernetes/kubelet-ca.crt"
May 28 08:12:02.009740 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.009713    2577 log.go:25] "Validated CRI v1 runtime API"
May 28 08:12:02.016231 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.016206    2577 log.go:25] "Validated CRI v1 image API"
May 28 08:12:02.041196 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.041169    2577 server.go:1452] "Using cgroup driver setting received from the CRI runtime" cgroupDriver="systemd"
May 28 08:12:02.043997 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.043973    2577 fs.go:135] Filesystem UUIDs: map[0c964c7f-4dce-49f2-ab28-1176f0b0412a:/dev/nvme0n1p3 7B77-95E7:/dev/nvme0n1p2 d7e39d12-f219-432b-97f7-6c05b90de486:/dev/nvme0n1p4]
May 28 08:12:02.044082 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.043995    2577 fs.go:136] Filesystem partitions: map[/dev/nvme0n1p3:{mountpoint:/boot major:259 minor:3 fsType:ext4 blockSize:0} /dev/nvme0n1p4:{mountpoint:/var major:259 minor:4 fsType:xfs blockSize:0} /dev/shm:{mountpoint:/dev/shm major:0 minor:22 fsType:tmpfs blockSize:0} /run:{mountpoint:/run major:0 minor:24 fsType:tmpfs blockSize:0} /tmp:{mountpoint:/tmp major:0 minor:31 fsType:tmpfs blockSize:0} composefs_0-33:{mountpoint:/ major:0 minor:33 fsType:overlay blockSize:0}]
May 28 08:12:02.049078 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.049028    2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kube-apiserver-client-kubelet" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 08:12:02.049078 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.049056    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jckll"
May 28 08:12:02.049606 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.049495    2577 manager.go:217] Machine: {Timestamp:2026-05-28 08:12:02.048440852 +0000 UTC m=+0.484620011 CPUVendorID:GenuineIntel NumCores:8 NumPhysicalCores:4 NumSockets:1 CpuFrequency:3098908 MemoryCapacity:32812167168 SwapCapacity:0 MemoryByType:map[] NVMInfo:{MemoryModeCapacity:0 AppDirectModeCapacity:0 AvgPowerBudget:0} HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] MachineID:ec2d0a8766f79256b5c92f4de39d3da0 SystemUUID:ec2d0a87-66f7-9256-b5c9-2f4de39d3da0 BootID:e63fc74b-7515-4f03-aef3-71b0218f5321 Filesystems:[{Device:/run DeviceMajor:0 DeviceMinor:24 Capacity:6562435072 Type:vfs Inodes:819200 HasInodes:true} {Device:/dev/nvme0n1p4 DeviceMajor:259 DeviceMinor:4 Capacity:128243970048 Type:vfs Inodes:62651840 HasInodes:true} {Device:composefs_0-33 DeviceMajor:0 DeviceMinor:33 Capacity:6103040 Type:vfs Inodes:18446744073709551615 HasInodes:true} {Device:/tmp DeviceMajor:0 DeviceMinor:31 Capacity:16406085632 Type:vfs Inodes:1048576 HasInodes:true} {Device:/dev/nvme0n1p3 DeviceMajor:259 DeviceMinor:3 Capacity:366869504 Type:vfs Inodes:98304 HasInodes:true} {Device:/dev/shm DeviceMajor:0 DeviceMinor:22 Capacity:16406081536 Type:vfs Inodes:4005391 HasInodes:true}] DiskMap:map[259:0:{Name:nvme0n1 Major:259 Minor:0 Size:128849018880 Scheduler:none}] NetworkDevices:[{Name:br-ex MacAddress:02:5e:b3:c9:03:63 Speed:0 Mtu:9001} {Name:ens5 MacAddress:02:5e:b3:c9:03:63 Speed:0 Mtu:9001} {Name:ovs-system MacAddress:e6:4d:e8:ed:db:1c Speed:0 Mtu:1500}] Topology:[{Id:0 Memory:32812167168 HugePages:[{PageSize:1048576 NumPages:0} {PageSize:2048 NumPages:0}] Cores:[{Id:0 Threads:[0 4] Caches:[{Id:0 Size:32768 Type:Data Level:1} {Id:0 Size:32768 Type:Instruction Level:1} {Id:0 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:1 Threads:[1 5] Caches:[{Id:1 Size:32768 Type:Data Level:1} {Id:1 Size:32768 Type:Instruction Level:1} {Id:1 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:2 Threads:[2 6] Caches:[{Id:2 Size:32768 Type:Data Level:1} {Id:2 Size:32768 Type:Instruction Level:1} {Id:2 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:} {Id:3 Threads:[3 7] Caches:[{Id:3 Size:32768 Type:Data Level:1} {Id:3 Size:32768 Type:Instruction Level:1} {Id:3 Size:1048576 Type:Unified Level:2}] UncoreCaches:[] SocketID:0 BookID: DrawerID:}] Caches:[{Id:0 Size:34603008 Type:Unified Level:3}] Distances:[10]}] CloudProvider:Unknown InstanceType:Unknown InstanceID:None}
May 28 08:12:02.050340 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.050328    2577 manager_no_libpfm.go:29] cAdvisor is build without cgo and/or libpfm support. Perf event counters are not available.
May 28 08:12:02.050481 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.050468    2577 manager.go:233] Version: {KernelVersion:5.14.0-570.114.1.el9_6.x86_64 ContainerOsVersion:Red Hat Enterprise Linux CoreOS 9.6.20260512-0 (Plow) DockerVersion: DockerAPIVersion: CadvisorVersion: CadvisorRevision:}
May 28 08:12:02.050855 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.050841    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kube-apiserver-client-kubelet" csr="csr-jckll"
May 28 08:12:02.051795 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.051764    2577 container_manager_linux.go:270] "Container manager verified user specified cgroup-root exists" cgroupRoot=[]
May 28 08:12:02.051929 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.051796    2577 container_manager_linux.go:275] "Creating Container Manager object based on Node Config" nodeConfig={"NodeName":"ip-10-0-137-158.ec2.internal","RuntimeCgroupsName":"/system.slice/crio.service","SystemCgroupsName":"/system.slice","KubeletCgroupsName":"","KubeletOOMScoreAdj":-999,"ContainerRuntime":"","CgroupsPerQOS":true,"CgroupRoot":"/","CgroupDriver":"systemd","KubeletRootDir":"/var/lib/kubelet","ProtectKernelDefaults":true,"KubeReservedCgroupName":"","SystemReservedCgroupName":"","ReservedSystemCPUs":{},"EnforceNodeAllocatable":{"pods":{}},"KubeReserved":null,"SystemReserved":{"cpu":"500m","ephemeral-storage":"1Gi","memory":"1Gi"},"HardEvictionThresholds":[{"Signal":"imagefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.15},"GracePeriod":0,"MinReclaim":null},{"Signal":"imagefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null},{"Signal":"memory.available","Operator":"LessThan","Value":{"Quantity":"100Mi","Percentage":0},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.available","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.1},"GracePeriod":0,"MinReclaim":null},{"Signal":"nodefs.inodesFree","Operator":"LessThan","Value":{"Quantity":null,"Percentage":0.05},"GracePeriod":0,"MinReclaim":null}],"QOSReserved":{},"CPUManagerPolicy":"none","CPUManagerPolicyOptions":null,"TopologyManagerScope":"container","CPUManagerReconcilePeriod":10000000000,"MemoryManagerPolicy":"None","MemoryManagerReservedMemory":null,"PodPidsLimit":4096,"EnforceCPULimits":true,"CPUCFSQuotaPeriod":100000000,"TopologyManagerPolicy":"none","TopologyManagerPolicyOptions":null,"CgroupVersion":2}
May 28 08:12:02.051973 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.051940    2577 topology_manager.go:138] "Creating topology manager with none policy"
May 28 08:12:02.051973 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.051949    2577 container_manager_linux.go:306] "Creating device plugin manager"
May 28 08:12:02.051973 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.051957    2577 manager.go:141] "Creating Device Plugin manager" path="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 08:12:02.052864 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.052853    2577 server.go:72] "Creating device plugin registration server" version="v1beta1" socket="/var/lib/kubelet/device-plugins/kubelet.sock"
May 28 08:12:02.054802 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.054791    2577 state_mem.go:36] "Initialized new in-memory state store"
May 28 08:12:02.055200 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.054902    2577 server.go:1267] "Using root directory" path="/var/lib/kubelet"
May 28 08:12:02.058023 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.058010    2577 kubelet.go:491] "Attempting to sync node with API server"
May 28 08:12:02.058085 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.058025    2577 kubelet.go:386] "Adding static pod path" path="/etc/kubernetes/manifests"
May 28 08:12:02.058085 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.058037    2577 file.go:69] "Watching path" path="/etc/kubernetes/manifests"
May 28 08:12:02.058085 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.058046    2577 kubelet.go:397] "Adding apiserver pod source"
May 28 08:12:02.058085 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.058056    2577 apiserver.go:42] "Waiting for node sync before watching apiserver pods"
May 28 08:12:02.059411 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.059397    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 08:12:02.059472 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.059418    2577 state_mem.go:40] "Initialized new in-memory state store for pod resource information tracking"
May 28 08:12:02.065019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.064996    2577 kuberuntime_manager.go:279] "Container runtime initialized" containerRuntime="cri-o" version="1.33.11-2.rhaos4.20.gitb2a8320.el9" apiVersion="v1"
May 28 08:12:02.067398 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.067379    2577 kubelet.go:953] "Not starting ClusterTrustBundle informer because we are in static kubelet mode or the ClusterTrustBundleProjection featuregate is disabled"
May 28 08:12:02.068433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068417    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/portworx-volume"
May 28 08:12:02.068504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068439    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/empty-dir"
May 28 08:12:02.068504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068447    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/git-repo"
May 28 08:12:02.068504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068455    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/host-path"
May 28 08:12:02.068504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068467    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/nfs"
May 28 08:12:02.068504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068475    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/secret"
May 28 08:12:02.068504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068484    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/iscsi"
May 28 08:12:02.068504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068492    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/downward-api"
May 28 08:12:02.068504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068503    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/fc"
May 28 08:12:02.068745 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068512    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/configmap"
May 28 08:12:02.068745 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068526    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/projected"
May 28 08:12:02.068966 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068955    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/local-volume"
May 28 08:12:02.069020 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.068997    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/csi"
May 28 08:12:02.069020 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.069007    2577 plugins.go:616] "Loaded volume plugin" pluginName="kubernetes.io/image"
May 28 08:12:02.072403 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.072386    2577 watchdog_linux.go:99] "Systemd watchdog is not enabled"
May 28 08:12:02.072488 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.072429    2577 server.go:1295] "Started kubelet"
May 28 08:12:02.072532 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.072480    2577 server.go:180] "Starting to listen" address="0.0.0.0" port=10250
May 28 08:12:02.072595 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.072544    2577 ratelimit.go:55] "Setting rate limiting for endpoint" service="podresources" qps=100 burstTokens=10
May 28 08:12:02.072707 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.072623    2577 server_v1.go:47] "podresources" method="list" useActivePods=true
May 28 08:12:02.073215 ip-10-0-137-158 systemd[1]: Started Kubernetes Kubelet.
May 28 08:12:02.074689 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.074649    2577 server.go:255] "Starting to serve the podresources API" endpoint="unix:/var/lib/kubelet/pod-resources/kubelet.sock"
May 28 08:12:02.074771 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.074732    2577 reflector.go:430] "Caches populated" type="*v1.Node" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 08:12:02.075014 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.075001    2577 server.go:317] "Adding debug handlers to kubelet server"
May 28 08:12:02.077294 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.077273    2577 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 08:12:02.083384 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.083363    2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-158.ec2.internal" not found
May 28 08:12:02.084765 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.084750    2577 fs_resource_analyzer.go:67] "Starting FS ResourceAnalyzer"
May 28 08:12:02.084892 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.084834    2577 certificate_manager.go:422] "Certificate rotation is enabled" logger="kubernetes.io/kubelet-serving"
May 28 08:12:02.085472 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.085100    2577 volume_manager.go:295] "The desired_state_of_world populator starts"
May 28 08:12:02.085472 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.085464    2577 volume_manager.go:297] "Starting Kubelet Volume Manager"
May 28 08:12:02.085635 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.085107    2577 desired_state_of_world_populator.go:150] "Desired state populator starts to run"
May 28 08:12:02.085635 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:02.085390    2577 kubelet.go:1618] "Image garbage collection failed once. Stats initialization may not have completed yet" err="failed to get imageFs info: unable to find data in memory cache"
May 28 08:12:02.085741 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.085707    2577 reconstruct.go:97] "Volume reconstruction finished"
May 28 08:12:02.085741 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.085718    2577 reconciler.go:26] "Reconciler: start to sync state"
May 28 08:12:02.086070 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.086055    2577 factory.go:55] Registering systemd factory
May 28 08:12:02.086070 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.086072    2577 factory.go:223] Registration of the systemd container factory successfully
May 28 08:12:02.086349 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.086330    2577 factory.go:153] Registering CRI-O factory
May 28 08:12:02.086349 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.086350    2577 factory.go:223] Registration of the crio container factory successfully
May 28 08:12:02.086499 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.086408    2577 factory.go:221] Registration of the containerd container factory failed: unable to create containerd client: containerd: cannot unix dial containerd api service: dial unix /run/containerd/containerd.sock: connect: no such file or directory
May 28 08:12:02.086499 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.086426    2577 factory.go:103] Registering Raw factory
May 28 08:12:02.086499 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.086443    2577 manager.go:1196] Started watching for new ooms in manager
May 28 08:12:02.087058 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.087030    2577 manager.go:319] Starting recovery of all containers
May 28 08:12:02.087197 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:02.087036    2577 kubelet_node_status.go:515] "Error getting the current node from lister" err="node \"ip-10-0-137-158.ec2.internal\" not found"
May 28 08:12:02.091504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.091332    2577 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 08:12:02.096071 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:02.095938    2577 nodelease.go:49] "Failed to get node when trying to set owner ref to the node lease" err="nodes \"ip-10-0-137-158.ec2.internal\" not found" node="ip-10-0-137-158.ec2.internal"
May 28 08:12:02.099735 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.099691    2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-158.ec2.internal" not found
May 28 08:12:02.100728 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.100713    2577 manager.go:324] Recovery completed
May 28 08:12:02.105171 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.105158    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 08:12:02.108422 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.108406    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-158.ec2.internal" event="NodeHasSufficientMemory"
May 28 08:12:02.108485 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.108434    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-158.ec2.internal" event="NodeHasNoDiskPressure"
May 28 08:12:02.108485 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.108445    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-158.ec2.internal" event="NodeHasSufficientPID"
May 28 08:12:02.108974 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.108960    2577 cpu_manager.go:222] "Starting CPU manager" policy="none"
May 28 08:12:02.108974 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.108972    2577 cpu_manager.go:223] "Reconciling" reconcilePeriod="10s"
May 28 08:12:02.109068 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.108999    2577 state_mem.go:36] "Initialized new in-memory state store"
May 28 08:12:02.112754 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.112741    2577 policy_none.go:49] "None policy: Start"
May 28 08:12:02.112799 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.112759    2577 memory_manager.go:186] "Starting memorymanager" policy="None"
May 28 08:12:02.112799 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.112770    2577 state_mem.go:35] "Initializing new in-memory state store"
May 28 08:12:02.153291 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.153273    2577 manager.go:341] "Starting Device Plugin manager"
May 28 08:12:02.176969 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:02.153315    2577 manager.go:517] "Failed to read data from checkpoint" err="checkpoint is not found" checkpoint="kubelet_internal_checkpoint"
May 28 08:12:02.176969 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.153328    2577 server.go:85] "Starting device plugin registration server"
May 28 08:12:02.176969 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.153637    2577 eviction_manager.go:189] "Eviction manager: starting control loop"
May 28 08:12:02.176969 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.153649    2577 container_log_manager.go:189] "Initializing container log rotate workers" workers=1 monitorPeriod="10s"
May 28 08:12:02.176969 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.153773    2577 plugin_watcher.go:51] "Plugin Watcher Start" path="/var/lib/kubelet/plugins_registry"
May 28 08:12:02.176969 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.153847    2577 plugin_manager.go:116] "The desired_state_of_world populator (plugin watcher) starts"
May 28 08:12:02.176969 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.153856    2577 plugin_manager.go:118] "Starting Kubelet Plugin Manager"
May 28 08:12:02.176969 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:02.154535    2577 eviction_manager.go:267] "eviction manager: failed to check if we have separate container filesystem. Ignoring." err="non-existent label \"crio-containers\""
May 28 08:12:02.176969 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:02.154569    2577 eviction_manager.go:292] "Eviction manager: failed to get summary stats" err="failed to get node info: node \"ip-10-0-137-158.ec2.internal\" not found"
May 28 08:12:02.176969 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.160640    2577 nodeinfomanager.go:417] Failed to publish CSINode: nodes "ip-10-0-137-158.ec2.internal" not found
May 28 08:12:02.235487 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.235393    2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv4"
May 28 08:12:02.236773 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.236759    2577 kubelet_network_linux.go:49] "Initialized iptables rules." protocol="IPv6"
May 28 08:12:02.236867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.236780    2577 status_manager.go:230] "Starting to sync pod status with apiserver"
May 28 08:12:02.236867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.236824    2577 watchdog_linux.go:127] "Systemd watchdog is not enabled or the interval is invalid, so health checking will not be started."
May 28 08:12:02.236867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.236834    2577 kubelet.go:2451] "Starting kubelet main sync loop"
May 28 08:12:02.236992 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:02.236882    2577 kubelet.go:2475] "Skipping pod synchronization" err="PLEG is not healthy: pleg has yet to be successful"
May 28 08:12:02.239886 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.239868    2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 08:12:02.254025 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.253989    2577 kubelet_node_status.go:413] "Setting node annotation to enable volume controller attach/detach"
May 28 08:12:02.255118 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.255100    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-158.ec2.internal" event="NodeHasSufficientMemory"
May 28 08:12:02.255218 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.255136    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-158.ec2.internal" event="NodeHasNoDiskPressure"
May 28 08:12:02.255218 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.255151    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-158.ec2.internal" event="NodeHasSufficientPID"
May 28 08:12:02.255218 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.255183    2577 kubelet_node_status.go:78] "Attempting to register node" node="ip-10-0-137-158.ec2.internal"
May 28 08:12:02.266735 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.266709    2577 kubelet_node_status.go:81] "Successfully registered node" node="ip-10-0-137-158.ec2.internal"
May 28 08:12:02.337121 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.337084    2577 kubelet.go:2537] "SyncLoop ADD" source="file" pods=["openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal","kube-system/kube-apiserver-proxy-ip-10-0-137-158.ec2.internal"]
May 28 08:12:02.339157 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.339137    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.339245 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.339142    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.355264 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.355238    2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.359771 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.359753    2577 kubelet.go:3340] "Creating a mirror pod for static pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.367102 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.367078    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 08:12:02.375041 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.375023    2577 warnings.go:110] "Warning: metadata.name: this is used in the Pod's hostname, which can result in surprising behavior; a DNS label is recommended: [must not contain dots]"
May 28 08:12:02.387230 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.387203    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/47909bb01acf11b78eb7bcc9fa0b9efc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal\" (UID: \"47909bb01acf11b78eb7bcc9fa0b9efc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.387230 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.387233    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47909bb01acf11b78eb7bcc9fa0b9efc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal\" (UID: \"47909bb01acf11b78eb7bcc9fa0b9efc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.387412 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.387251    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ac83f6a6291c1bd1f8450f6fd7149beb-config\") pod \"kube-apiserver-proxy-ip-10-0-137-158.ec2.internal\" (UID: \"ac83f6a6291c1bd1f8450f6fd7149beb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.488263 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.488181    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ac83f6a6291c1bd1f8450f6fd7149beb-config\") pod \"kube-apiserver-proxy-ip-10-0-137-158.ec2.internal\" (UID: \"ac83f6a6291c1bd1f8450f6fd7149beb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.488263 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.488210    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/47909bb01acf11b78eb7bcc9fa0b9efc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal\" (UID: \"47909bb01acf11b78eb7bcc9fa0b9efc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.488263 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.488228    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47909bb01acf11b78eb7bcc9fa0b9efc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal\" (UID: \"47909bb01acf11b78eb7bcc9fa0b9efc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.488467 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.488279    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/47909bb01acf11b78eb7bcc9fa0b9efc-var-lib-kubelet\") pod \"kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal\" (UID: \"47909bb01acf11b78eb7bcc9fa0b9efc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.488467 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.488290    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/host-path/ac83f6a6291c1bd1f8450f6fd7149beb-config\") pod \"kube-apiserver-proxy-ip-10-0-137-158.ec2.internal\" (UID: \"ac83f6a6291c1bd1f8450f6fd7149beb\") " pod="kube-system/kube-apiserver-proxy-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.488467 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.488320    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kube\" (UniqueName: \"kubernetes.io/host-path/47909bb01acf11b78eb7bcc9fa0b9efc-etc-kube\") pod \"kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal\" (UID: \"47909bb01acf11b78eb7bcc9fa0b9efc\") " pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.669603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.669564    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.677344 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.677320    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal"
May 28 08:12:02.961859 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.961766    2577 transport.go:147] "Certificate rotation detected, shutting down client connections to start using new credentials"
May 28 08:12:02.962541 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.961930    2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.CSIDriver" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 08:12:02.962541 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.961959    2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.RuntimeClass" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 08:12:02.962541 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:02.961980    2577 reflector.go:556] "Warning: watch ended with error" reflector="k8s.io/client-go/informers/factory.go:160" type="*v1.Service" err="very short watch: k8s.io/client-go/informers/factory.go:160: Unexpected watch close - watch lasted less than a second and no items received"
May 28 08:12:03.053226 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.053178    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kube-apiserver-client-kubelet" expiration="2028-05-27 08:07:02 +0000 UTC" deadline="2027-11-17 00:42:47.275725527 +0000 UTC"
May 28 08:12:03.053226 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.053215    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kube-apiserver-client-kubelet" sleep="12904h30m44.222512942s"
May 28 08:12:03.058355 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.058323    2577 apiserver.go:52] "Watching apiserver"
May 28 08:12:03.066839 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.066814    2577 reflector.go:430] "Caches populated" type="*v1.Pod" reflector="pkg/kubelet/config/apiserver.go:66"
May 28 08:12:03.067188 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.067164    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/kube-apiserver-proxy-ip-10-0-137-158.ec2.internal","openshift-image-registry/node-ca-bp4g8","openshift-multus/multus-additional-cni-plugins-8vp6h","openshift-multus/multus-mbdnp","openshift-multus/network-metrics-daemon-mlpsc","openshift-network-operator/iptables-alerter-p6k96","kube-system/konnectivity-agent-j8bt4","openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch","openshift-cluster-node-tuning-operator/tuned-fhw4w","openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal","openshift-network-diagnostics/network-check-target-nkgkq","openshift-ovn-kubernetes/ovnkube-node-hpqw6"]
May 28 08:12:03.069359 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.069337    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bp4g8"
May 28 08:12:03.070216 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.070184    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.070396 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.070373    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.071173 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.071153    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:03.071255 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.071232    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:03.072358 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.072085    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p6k96"
May 28 08:12:03.072358 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.072125    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"image-registry-certificates\""
May 28 08:12:03.072358 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.072174    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"kube-root-ca.crt\""
May 28 08:12:03.072358 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.072181    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"node-ca-dockercfg-mdl74\""
May 28 08:12:03.072358 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.072126    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"openshift-service-ca.crt\""
May 28 08:12:03.072593 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.072571    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"multus-ancillary-tools-dockercfg-z96qb\""
May 28 08:12:03.072886 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.072869    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j8bt4"
May 28 08:12:03.073276 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.073256    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"openshift-service-ca.crt\""
May 28 08:12:03.073388 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.073317    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"cni-copy-resources\""
May 28 08:12:03.073744 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.073715    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"default-cni-sysctl-allowlist\""
May 28 08:12:03.073744 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.073725    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"whereabouts-flatfile-config\""
May 28 08:12:03.073896 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.073718    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"multus-daemon-config\""
May 28 08:12:03.073896 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.073851    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-multus\"/\"kube-root-ca.crt\""
May 28 08:12:03.073896 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.073866    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"default-dockercfg-crjnw\""
May 28 08:12:03.074408 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.074393    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"openshift-service-ca.crt\""
May 28 08:12:03.074816 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.074797    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"kube-root-ca.crt\""
May 28 08:12:03.074905 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.074890    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.075157 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.075055    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-script\""
May 28 08:12:03.075157 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.075076    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-operator\"/\"iptables-alerter-dockercfg-nrs2m\""
May 28 08:12:03.075754 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.075739    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"kube-system\"/\"konnectivity-ca-bundle\""
May 28 08:12:03.075844 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.075798    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.076279 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.076237    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"konnectivity-agent\""
May 28 08:12:03.076601 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.076579    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"default-dockercfg-zd7fs\""
May 28 08:12:03.076783 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.076767    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:03.076917 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.076895    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:03.077867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.077844    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"openshift-service-ca.crt\""
May 28 08:12:03.078137 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.078097    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-metrics-serving-cert\""
May 28 08:12:03.078534 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.078517    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-csi-drivers\"/\"kube-root-ca.crt\""
May 28 08:12:03.078739 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.078716    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.078924 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.078908    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-csi-drivers\"/\"aws-ebs-csi-driver-node-sa-dockercfg-mzlg6\""
May 28 08:12:03.079917 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.079900    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"kube-root-ca.crt\""
May 28 08:12:03.080008 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.079950    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"tuned-dockercfg-ldk9w\""
May 28 08:12:03.080008 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.079977    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-cluster-node-tuning-operator\"/\"openshift-service-ca.crt\""
May 28 08:12:03.081094 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.081075    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-config\""
May 28 08:12:03.081167 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.081126    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"env-overrides\""
May 28 08:12:03.081355 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.081337    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"kube-root-ca.crt\""
May 28 08:12:03.081436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.081361    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"openshift-service-ca.crt\""
May 28 08:12:03.082732 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.082712    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-node-metrics-cert\""
May 28 08:12:03.082817 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.082712    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ovn-kubernetes\"/\"ovn-kubernetes-node-dockercfg-g47r5\""
May 28 08:12:03.082905 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.082891    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ovn-kubernetes\"/\"ovnkube-script-lib\""
May 28 08:12:03.085959 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.085689    2577 certificate_manager.go:566] "Rotating certificates" logger="kubernetes.io/kubelet-serving"
May 28 08:12:03.086547 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.086530    2577 desired_state_of_world_populator.go:158] "Finished populating initial desired state of world"
May 28 08:12:03.090801 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.090781    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mh8pn\" (UniqueName: \"kubernetes.io/projected/dca0512e-79d3-4c51-9f05-cc6c64fe4f85-kube-api-access-mh8pn\") pod \"iptables-alerter-p6k96\" (UID: \"dca0512e-79d3-4c51-9f05-cc6c64fe4f85\") " pod="openshift-network-operator/iptables-alerter-p6k96"
May 28 08:12:03.090914 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.090809    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-sysctl-d\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.090914 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.090832    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-run\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.090914 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.090854    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-host\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.090914 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.090878    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-os-release\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.090914 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.090895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-etc-selinux\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.090914 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.090910    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-slash\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.091221 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.090995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-fwkdw\" (UniqueName: \"kubernetes.io/projected/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-kube-api-access-fwkdw\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.091221 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-kubelet\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.091221 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091056    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-cni-netd\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.091221 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091080    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21817e48-7f6c-4c5b-bd3c-ada88d6de024-env-overrides\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.091221 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/80faa7ab-5aca-4d3f-96cf-f7edc87643fe-agent-certs\") pod \"konnectivity-agent-j8bt4\" (UID: \"80faa7ab-5aca-4d3f-96cf-f7edc87643fe\") " pod="kube-system/konnectivity-agent-j8bt4"
May 28 08:12:03.091221 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091126    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-kubernetes\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.091221 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-run-netns\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.091221 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091172    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-device-dir\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.091221 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091197    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21817e48-7f6c-4c5b-bd3c-ada88d6de024-ovnkube-config\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-cnibin\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091245    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/091802e9-2e2c-4c10-a42c-88623f528cd0-multus-daemon-config\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091272    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-etc-openvswitch\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091342    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dca0512e-79d3-4c51-9f05-cc6c64fe4f85-host-slash\") pod \"iptables-alerter-p6k96\" (UID: \"dca0512e-79d3-4c51-9f05-cc6c64fe4f85\") " pod="openshift-network-operator/iptables-alerter-p6k96"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091367    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-systemd\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091390    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-etc-kubernetes\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091413    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-vggdr\" (UniqueName: \"kubernetes.io/projected/091802e9-2e2c-4c10-a42c-88623f528cd0-kube-api-access-vggdr\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091450    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-log-socket\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091471    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-hostroot\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091514    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-system-cni-dir\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091539    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-multus-cni-dir\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091561    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-sys-fs\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091587    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21817e48-7f6c-4c5b-bd3c-ada88d6de024-ovn-node-metrics-cert\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091635    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-cnibin\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091686    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/80faa7ab-5aca-4d3f-96cf-f7edc87643fe-konnectivity-ca\") pod \"konnectivity-agent-j8bt4\" (UID: \"80faa7ab-5aca-4d3f-96cf-f7edc87643fe\") " pod="kube-system/konnectivity-agent-j8bt4"
May 28 08:12:03.091700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091705    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-lib-modules\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091735    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kt796\" (UniqueName: \"kubernetes.io/projected/18155ab5-6646-4bf6-88af-c74297a72a1e-kube-api-access-kt796\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091757    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/091802e9-2e2c-4c10-a42c-88623f528cd0-cni-binary-copy\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-var-lib-cni-bin\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-multus-conf-dir\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091811    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-run-multus-certs\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091826    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-cni-binary-copy\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091844    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-sysconfig\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091864    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-socket-dir\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091879    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091894    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-sys\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091920    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-var-lib-cni-multus\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091947    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091967    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-systemd-units\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.091992    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-node-log\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6pdl\" (UniqueName: \"kubernetes.io/projected/a4b772f7-1b71-4e3b-b2a9-fbf33edfce66-kube-api-access-j6pdl\") pod \"node-ca-bp4g8\" (UID: \"a4b772f7-1b71-4e3b-b2a9-fbf33edfce66\") " pod="openshift-image-registry/node-ca-bp4g8"
May 28 08:12:03.092387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-modprobe-d\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092044    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-system-cni-dir\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092067    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-multus-socket-dir-parent\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092089    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-run-openvswitch\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092103    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a4b772f7-1b71-4e3b-b2a9-fbf33edfce66-serviceca\") pod \"node-ca-bp4g8\" (UID: \"a4b772f7-1b71-4e3b-b2a9-fbf33edfce66\") " pod="openshift-image-registry/node-ca-bp4g8"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092119    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-var-lib-kubelet\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092194    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-run-k8s-cni-cncf-io\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092232    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-registration-dir\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092257    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-run-systemd\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092283    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-cni-bin\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092310    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092335    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092365    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-run-netns\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092387    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21817e48-7f6c-4c5b-bd3c-ada88d6de024-ovnkube-script-lib\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092409    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6s66m\" (UniqueName: \"kubernetes.io/projected/90e9cadb-328a-47e1-a119-bae6987ac229-kube-api-access-6s66m\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:03.092987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092432    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092460    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-z68nq\" (UniqueName: \"kubernetes.io/projected/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-kube-api-access-z68nq\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092483    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dca0512e-79d3-4c51-9f05-cc6c64fe4f85-iptables-alerter-script\") pod \"iptables-alerter-p6k96\" (UID: \"dca0512e-79d3-4c51-9f05-cc6c64fe4f85\") " pod="openshift-network-operator/iptables-alerter-p6k96"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092512    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-sysctl-conf\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092542    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-var-lib-kubelet\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092591    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jbb2d\" (UniqueName: \"kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d\") pod \"network-check-target-nkgkq\" (UID: \"ecd64268-adf3-40c0-aaaf-510d4c689d8c\") " pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092616    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-os-release\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092634    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-tuned\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092694    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-var-lib-openvswitch\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092714    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-run-ovn-kubernetes\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092728    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cq9tc\" (UniqueName: \"kubernetes.io/projected/21817e48-7f6c-4c5b-bd3c-ada88d6de024-kube-api-access-cq9tc\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092741    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4b772f7-1b71-4e3b-b2a9-fbf33edfce66-host\") pod \"node-ca-bp4g8\" (UID: \"a4b772f7-1b71-4e3b-b2a9-fbf33edfce66\") " pod="openshift-image-registry/node-ca-bp4g8"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092762    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18155ab5-6646-4bf6-88af-c74297a72a1e-tmp\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.093417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.092796    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-run-ovn\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.097313 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.097293    2577 reflector.go:430] "Caches populated" logger="kubernetes.io/kubelet-serving" type="*v1.CertificateSigningRequest" reflector="k8s.io/client-go/tools/watch/informerwatcher.go:162"
May 28 08:12:03.117605 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.117574    2577 csr.go:274] "Certificate signing request is approved, waiting to be issued" logger="kubernetes.io/kubelet-serving" csr="csr-42prx"
May 28 08:12:03.132309 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.132273    2577 csr.go:270] "Certificate signing request is issued" logger="kubernetes.io/kubelet-serving" csr="csr-42prx"
May 28 08:12:03.193054 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193028    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-kubernetes\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.193205 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193059    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-run-netns\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.193205 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193079    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-device-dir\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.193205 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193102    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21817e48-7f6c-4c5b-bd3c-ada88d6de024-ovnkube-config\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.193205 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193140    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-kubernetes\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.193205 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193177    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"device-dir\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-device-dir\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.193433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193226    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-cnibin\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.193433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193255    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/091802e9-2e2c-4c10-a42c-88623f528cd0-multus-daemon-config\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.193433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193278    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-etc-openvswitch\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.193433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193300    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dca0512e-79d3-4c51-9f05-cc6c64fe4f85-host-slash\") pod \"iptables-alerter-p6k96\" (UID: \"dca0512e-79d3-4c51-9f05-cc6c64fe4f85\") " pod="openshift-network-operator/iptables-alerter-p6k96"
May 28 08:12:03.193433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193326    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-systemd\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.193433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193344    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-etc-kubernetes\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.193433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193360    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-vggdr\" (UniqueName: \"kubernetes.io/projected/091802e9-2e2c-4c10-a42c-88623f528cd0-kube-api-access-vggdr\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.193433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193378    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-log-socket\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.193433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193401    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-hostroot\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.193433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193424    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-system-cni-dir\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.193889 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193447    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-multus-cni-dir\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.193889 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193469    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-sys-fs\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.193889 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193494    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21817e48-7f6c-4c5b-bd3c-ada88d6de024-ovn-node-metrics-cert\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.193889 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193518    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-cnibin\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.193889 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193524    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-kubernetes\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-etc-kubernetes\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.193889 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193542    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/80faa7ab-5aca-4d3f-96cf-f7edc87643fe-konnectivity-ca\") pod \"konnectivity-agent-j8bt4\" (UID: \"80faa7ab-5aca-4d3f-96cf-f7edc87643fe\") " pod="kube-system/konnectivity-agent-j8bt4"
May 28 08:12:03.193889 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193571    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-lib-modules\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.193889 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193578    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-cnibin\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.193889 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193609    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-system-cni-dir\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.193889 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193691    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cnibin\" (UniqueName: \"kubernetes.io/host-path/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-cnibin\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.193889 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193701    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys-fs\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-sys-fs\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.193889 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193729    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-config\" (UniqueName: \"kubernetes.io/configmap/21817e48-7f6c-4c5b-bd3c-ada88d6de024-ovnkube-config\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193911    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-lib-modules\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193953    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/dca0512e-79d3-4c51-9f05-cc6c64fe4f85-host-slash\") pod \"iptables-alerter-p6k96\" (UID: \"dca0512e-79d3-4c51-9f05-cc6c64fe4f85\") " pod="openshift-network-operator/iptables-alerter-p6k96"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193984    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-etc-openvswitch\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194029    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-systemd\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-systemd\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194071    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hostroot\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-hostroot\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194095    2577 swap_util.go:74] "error creating dir to test if tmpfs noswap is enabled. Assuming not supported" mount path="" error="stat /var/lib/kubelet/plugins/kubernetes.io/empty-dir: no such file or directory"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194135    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-daemon-config\" (UniqueName: \"kubernetes.io/configmap/091802e9-2e2c-4c10-a42c-88623f528cd0-multus-daemon-config\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194149    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"konnectivity-ca\" (UniqueName: \"kubernetes.io/configmap/80faa7ab-5aca-4d3f-96cf-f7edc87643fe-konnectivity-ca\") pod \"konnectivity-agent-j8bt4\" (UID: \"80faa7ab-5aca-4d3f-96cf-f7edc87643fe\") " pod="kube-system/konnectivity-agent-j8bt4"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.193615    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kt796\" (UniqueName: \"kubernetes.io/projected/18155ab5-6646-4bf6-88af-c74297a72a1e-kube-api-access-kt796\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194190    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/091802e9-2e2c-4c10-a42c-88623f528cd0-cni-binary-copy\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194211    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-cni-dir\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-multus-cni-dir\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194216    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-var-lib-cni-bin\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194241    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-multus-conf-dir\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194266    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-run-multus-certs\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194277    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-bin\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-var-lib-cni-bin\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194244    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"log-socket\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-log-socket\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194291    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-cni-binary-copy\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.194414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194320    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-sysconfig\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194346    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-socket-dir\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194374    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194424    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-sys\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194434    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-multus-certs\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-run-multus-certs\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194453    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-var-lib-cni-multus\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194489    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194511    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-systemd-units\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194531    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-node-log\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194555    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6pdl\" (UniqueName: \"kubernetes.io/projected/a4b772f7-1b71-4e3b-b2a9-fbf33edfce66-kube-api-access-j6pdl\") pod \"node-ca-bp4g8\" (UID: \"a4b772f7-1b71-4e3b-b2a9-fbf33edfce66\") " pod="openshift-image-registry/node-ca-bp4g8"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194580    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-modprobe-d\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194604    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-system-cni-dir\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194621    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-multus\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-var-lib-cni-multus\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194628    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-multus-socket-dir-parent\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194675    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-run-openvswitch\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194689    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-conf-dir\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-multus-conf-dir\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194699    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a4b772f7-1b71-4e3b-b2a9-fbf33edfce66-serviceca\") pod \"node-ca-bp4g8\" (UID: \"a4b772f7-1b71-4e3b-b2a9-fbf33edfce66\") " pod="openshift-image-registry/node-ca-bp4g8"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194720    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/091802e9-2e2c-4c10-a42c-88623f528cd0-cni-binary-copy\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.195164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194725    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysconfig\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-sysconfig\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194763    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-binary-copy\" (UniqueName: \"kubernetes.io/configmap/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-cni-binary-copy\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194780    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"systemd-units\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-systemd-units\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194800    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"system-cni-dir\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-system-cni-dir\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194817    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"socket-dir\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-socket-dir\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194822    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-log\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-node-log\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194840    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-modprobe-d\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-modprobe-d\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194853    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"multus-socket-dir-parent\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-multus-socket-dir-parent\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194902    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-run-openvswitch\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194922    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-sys\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194722    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194996    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-var-lib-kubelet\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195021    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-run-k8s-cni-cncf-io\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195042    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-registration-dir\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.194554    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-run-netns\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195096    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registration-dir\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-registration-dir\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195105    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-run-systemd\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.196164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195154    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-var-lib-kubelet\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195180    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"whereabouts-flatfile-configmap\" (UniqueName: \"kubernetes.io/configmap/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-whereabouts-flatfile-configmap\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195191    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-k8s-cni-cncf-io\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-run-k8s-cni-cncf-io\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195057    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-systemd\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-run-systemd\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195218    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-cni-networks-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-var-lib-cni-networks-ovn-kubernetes\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195239    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-cni-bin\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195266    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195277    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serviceca\" (UniqueName: \"kubernetes.io/configmap/a4b772f7-1b71-4e3b-b2a9-fbf33edfce66-serviceca\") pod \"node-ca-bp4g8\" (UID: \"a4b772f7-1b71-4e3b-b2a9-fbf33edfce66\") " pod="openshift-image-registry/node-ca-bp4g8"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195283    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-bin\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-cni-bin\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195300    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195327    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-run-netns\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195351    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21817e48-7f6c-4c5b-bd3c-ada88d6de024-ovnkube-script-lib\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195374    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6s66m\" (UniqueName: \"kubernetes.io/projected/90e9cadb-328a-47e1-a119-bae6987ac229-kube-api-access-6s66m\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195390    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tuning-conf-dir\" (UniqueName: \"kubernetes.io/host-path/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-tuning-conf-dir\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195397    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195391    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-netns\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-run-netns\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195425    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-z68nq\" (UniqueName: \"kubernetes.io/projected/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-kube-api-access-z68nq\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.196958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195451    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dca0512e-79d3-4c51-9f05-cc6c64fe4f85-iptables-alerter-script\") pod \"iptables-alerter-p6k96\" (UID: \"dca0512e-79d3-4c51-9f05-cc6c64fe4f85\") " pod="openshift-network-operator/iptables-alerter-p6k96"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195480    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-sysctl-conf\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195504    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-var-lib-kubelet\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195529    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jbb2d\" (UniqueName: \"kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d\") pod \"network-check-target-nkgkq\" (UID: \"ecd64268-adf3-40c0-aaaf-510d4c689d8c\") " pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195554    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-os-release\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.195559    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195586    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-tuned\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195610    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-var-lib-openvswitch\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195688    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"var-lib-openvswitch\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-var-lib-openvswitch\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.195716    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs podName:90e9cadb-328a-47e1-a119-bae6987ac229 nodeName:}" failed. No retries permitted until 2026-05-28 08:12:03.695635434 +0000 UTC m=+2.131814594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs") pod "network-metrics-daemon-mlpsc" (UID: "90e9cadb-328a-47e1-a119-bae6987ac229") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195729    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-conf\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-sysctl-conf\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195740    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-run-ovn-kubernetes\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195806    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-os-release\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195813    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cq9tc\" (UniqueName: \"kubernetes.io/projected/21817e48-7f6c-4c5b-bd3c-ada88d6de024-kube-api-access-cq9tc\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195825    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cni-sysctl-allowlist\" (UniqueName: \"kubernetes.io/configmap/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-cni-sysctl-allowlist\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195858    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4b772f7-1b71-4e3b-b2a9-fbf33edfce66-host\") pod \"node-ca-bp4g8\" (UID: \"a4b772f7-1b71-4e3b-b2a9-fbf33edfce66\") " pod="openshift-image-registry/node-ca-bp4g8"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195882    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18155ab5-6646-4bf6-88af-c74297a72a1e-tmp\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.197867 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.195884    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-dir\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-kubelet-dir\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196059    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovnkube-script-lib\" (UniqueName: \"kubernetes.io/configmap/21817e48-7f6c-4c5b-bd3c-ada88d6de024-ovnkube-script-lib\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196062    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-run-ovn\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196098    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-run-ovn-kubernetes\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-run-ovn-kubernetes\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196116    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run-ovn\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-run-ovn\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196123    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"iptables-alerter-script\" (UniqueName: \"kubernetes.io/configmap/dca0512e-79d3-4c51-9f05-cc6c64fe4f85-iptables-alerter-script\") pod \"iptables-alerter-p6k96\" (UID: \"dca0512e-79d3-4c51-9f05-cc6c64fe4f85\") " pod="openshift-network-operator/iptables-alerter-p6k96"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196140    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/a4b772f7-1b71-4e3b-b2a9-fbf33edfce66-host\") pod \"node-ca-bp4g8\" (UID: \"a4b772f7-1b71-4e3b-b2a9-fbf33edfce66\") " pod="openshift-image-registry/node-ca-bp4g8"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196175    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-var-lib-kubelet\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-host-var-lib-kubelet\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196179    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mh8pn\" (UniqueName: \"kubernetes.io/projected/dca0512e-79d3-4c51-9f05-cc6c64fe4f85-kube-api-access-mh8pn\") pod \"iptables-alerter-p6k96\" (UID: \"dca0512e-79d3-4c51-9f05-cc6c64fe4f85\") " pod="openshift-network-operator/iptables-alerter-p6k96"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196212    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-sysctl-d\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196466    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-sysctl-d\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-sysctl-d\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196500    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"run\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-run\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196523    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-host\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196541    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-os-release\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196561    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-etc-selinux\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196582    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-slash\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196605    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-fwkdw\" (UniqueName: \"kubernetes.io/projected/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-kube-api-access-fwkdw\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196639    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-kubelet\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.198436 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196683    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-cni-netd\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196705    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21817e48-7f6c-4c5b-bd3c-ada88d6de024-env-overrides\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196729    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/80faa7ab-5aca-4d3f-96cf-f7edc87643fe-agent-certs\") pod \"konnectivity-agent-j8bt4\" (UID: \"80faa7ab-5aca-4d3f-96cf-f7edc87643fe\") " pod="kube-system/konnectivity-agent-j8bt4"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196771    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"run\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-run\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.196807    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-slash\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-slash\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.197008    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-kubelet\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-kubelet\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.197053    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host-cni-netd\" (UniqueName: \"kubernetes.io/host-path/21817e48-7f6c-4c5b-bd3c-ada88d6de024-host-cni-netd\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.197092    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-selinux\" (UniqueName: \"kubernetes.io/host-path/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-etc-selinux\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.197205    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"host\" (UniqueName: \"kubernetes.io/host-path/18155ab5-6646-4bf6-88af-c74297a72a1e-host\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.197272    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"os-release\" (UniqueName: \"kubernetes.io/host-path/091802e9-2e2c-4c10-a42c-88623f528cd0-os-release\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.197622    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"env-overrides\" (UniqueName: \"kubernetes.io/configmap/21817e48-7f6c-4c5b-bd3c-ada88d6de024-env-overrides\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.198163    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ovn-node-metrics-cert\" (UniqueName: \"kubernetes.io/secret/21817e48-7f6c-4c5b-bd3c-ada88d6de024-ovn-node-metrics-cert\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.198174    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"etc-tuned\" (UniqueName: \"kubernetes.io/empty-dir/18155ab5-6646-4bf6-88af-c74297a72a1e-etc-tuned\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.199028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.198271    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp\" (UniqueName: \"kubernetes.io/empty-dir/18155ab5-6646-4bf6-88af-c74297a72a1e-tmp\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.199389 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.199145    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"agent-certs\" (UniqueName: \"kubernetes.io/secret/80faa7ab-5aca-4d3f-96cf-f7edc87643fe-agent-certs\") pod \"konnectivity-agent-j8bt4\" (UID: \"80faa7ab-5aca-4d3f-96cf-f7edc87643fe\") " pod="kube-system/konnectivity-agent-j8bt4"
May 28 08:12:03.203861 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.203819    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 08:12:03.203861 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.203847    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 08:12:03.203861 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.203861    2577 projected.go:194] Error preparing data for projected volume kube-api-access-jbb2d for pod openshift-network-diagnostics/network-check-target-nkgkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:03.204367 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.204344    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d podName:ecd64268-adf3-40c0-aaaf-510d4c689d8c nodeName:}" failed. No retries permitted until 2026-05-28 08:12:03.704319298 +0000 UTC m=+2.140498465 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-api-access-jbb2d" (UniqueName: "kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d") pod "network-check-target-nkgkq" (UID: "ecd64268-adf3-40c0-aaaf-510d4c689d8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:03.205071 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.204913    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kt796\" (UniqueName: \"kubernetes.io/projected/18155ab5-6646-4bf6-88af-c74297a72a1e-kube-api-access-kt796\") pod \"tuned-fhw4w\" (UID: \"18155ab5-6646-4bf6-88af-c74297a72a1e\") " pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.205242 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.205218    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6pdl\" (UniqueName: \"kubernetes.io/projected/a4b772f7-1b71-4e3b-b2a9-fbf33edfce66-kube-api-access-j6pdl\") pod \"node-ca-bp4g8\" (UID: \"a4b772f7-1b71-4e3b-b2a9-fbf33edfce66\") " pod="openshift-image-registry/node-ca-bp4g8"
May 28 08:12:03.205623 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.205600    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-vggdr\" (UniqueName: \"kubernetes.io/projected/091802e9-2e2c-4c10-a42c-88623f528cd0-kube-api-access-vggdr\") pod \"multus-mbdnp\" (UID: \"091802e9-2e2c-4c10-a42c-88623f528cd0\") " pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.205911 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.205894    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6s66m\" (UniqueName: \"kubernetes.io/projected/90e9cadb-328a-47e1-a119-bae6987ac229-kube-api-access-6s66m\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:03.206181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.206165    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-z68nq\" (UniqueName: \"kubernetes.io/projected/48fe84ba-9a15-4a3d-a449-fb2694fbdfcf-kube-api-access-z68nq\") pod \"multus-additional-cni-plugins-8vp6h\" (UID: \"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf\") " pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.206854 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.206827    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-fwkdw\" (UniqueName: \"kubernetes.io/projected/752af6d4-3cd7-497c-9d85-0a1f6ab46f5f-kube-api-access-fwkdw\") pod \"aws-ebs-csi-driver-node-4v4ch\" (UID: \"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f\") " pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.207497 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.207477    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cq9tc\" (UniqueName: \"kubernetes.io/projected/21817e48-7f6c-4c5b-bd3c-ada88d6de024-kube-api-access-cq9tc\") pod \"ovnkube-node-hpqw6\" (UID: \"21817e48-7f6c-4c5b-bd3c-ada88d6de024\") " pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.207697 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.207674    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mh8pn\" (UniqueName: \"kubernetes.io/projected/dca0512e-79d3-4c51-9f05-cc6c64fe4f85-kube-api-access-mh8pn\") pod \"iptables-alerter-p6k96\" (UID: \"dca0512e-79d3-4c51-9f05-cc6c64fe4f85\") " pod="openshift-network-operator/iptables-alerter-p6k96"
May 28 08:12:03.217231 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.217171    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-node-tuning-operator/tuned-fhw4w"
May 28 08:12:03.222497 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.222474    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:03.281175 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:03.281140    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod47909bb01acf11b78eb7bcc9fa0b9efc.slice/crio-54c759d2e3f6aceec3beba13d592f3f97af53511047586d6a2f412e17fbff70c WatchSource:0}: Error finding container 54c759d2e3f6aceec3beba13d592f3f97af53511047586d6a2f412e17fbff70c: Status 404 returned error can't find the container with id 54c759d2e3f6aceec3beba13d592f3f97af53511047586d6a2f412e17fbff70c
May 28 08:12:03.281432 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:03.281408    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podac83f6a6291c1bd1f8450f6fd7149beb.slice/crio-b7288a074e71a5a2faf941b4fe97d13870cb17c49b1bdb9a230f914c910bafc6 WatchSource:0}: Error finding container b7288a074e71a5a2faf941b4fe97d13870cb17c49b1bdb9a230f914c910bafc6: Status 404 returned error can't find the container with id b7288a074e71a5a2faf941b4fe97d13870cb17c49b1bdb9a230f914c910bafc6
May 28 08:12:03.284600 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.284582    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 08:12:03.402296 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.402265    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/node-ca-bp4g8"
May 28 08:12:03.408499 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:03.408469    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poda4b772f7_1b71_4e3b_b2a9_fbf33edfce66.slice/crio-9b59ed8dd2e03f6f19df6f9ddd874ab400c2938141eb998f0019f35353f443c8 WatchSource:0}: Error finding container 9b59ed8dd2e03f6f19df6f9ddd874ab400c2938141eb998f0019f35353f443c8: Status 404 returned error can't find the container with id 9b59ed8dd2e03f6f19df6f9ddd874ab400c2938141eb998f0019f35353f443c8
May 28 08:12:03.416402 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.416383    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-additional-cni-plugins-8vp6h"
May 28 08:12:03.422725 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:03.422691    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48fe84ba_9a15_4a3d_a449_fb2694fbdfcf.slice/crio-6cfde6ef64661f6c9f4c5d6cb55e80b6490cf11ae88b5f0b28b7b45849717b4b WatchSource:0}: Error finding container 6cfde6ef64661f6c9f4c5d6cb55e80b6490cf11ae88b5f0b28b7b45849717b4b: Status 404 returned error can't find the container with id 6cfde6ef64661f6c9f4c5d6cb55e80b6490cf11ae88b5f0b28b7b45849717b4b
May 28 08:12:03.427843 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.427824    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/multus-mbdnp"
May 28 08:12:03.433817 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:03.433782    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod091802e9_2e2c_4c10_a42c_88623f528cd0.slice/crio-37fdc0185c48084b2a98cd9c5a163b27dfa7ed45652aac45a3c139366dc59768 WatchSource:0}: Error finding container 37fdc0185c48084b2a98cd9c5a163b27dfa7ed45652aac45a3c139366dc59768: Status 404 returned error can't find the container with id 37fdc0185c48084b2a98cd9c5a163b27dfa7ed45652aac45a3c139366dc59768
May 28 08:12:03.450187 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.450161    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-operator/iptables-alerter-p6k96"
May 28 08:12:03.457486 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:03.457456    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-poddca0512e_79d3_4c51_9f05_cc6c64fe4f85.slice/crio-3fd6eedc897c9e740c135e0aa6e12e6d4884814f943179a7c067aeb45511f518 WatchSource:0}: Error finding container 3fd6eedc897c9e740c135e0aa6e12e6d4884814f943179a7c067aeb45511f518: Status 404 returned error can't find the container with id 3fd6eedc897c9e740c135e0aa6e12e6d4884814f943179a7c067aeb45511f518
May 28 08:12:03.466696 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.466668    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/konnectivity-agent-j8bt4"
May 28 08:12:03.473037 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:03.473010    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod80faa7ab_5aca_4d3f_96cf_f7edc87643fe.slice/crio-133c024a3d5e8f580200f4fab6a43387adba2708367549d0643a1d499c53da35 WatchSource:0}: Error finding container 133c024a3d5e8f580200f4fab6a43387adba2708367549d0643a1d499c53da35: Status 404 returned error can't find the container with id 133c024a3d5e8f580200f4fab6a43387adba2708367549d0643a1d499c53da35
May 28 08:12:03.484206 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.484180    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch"
May 28 08:12:03.490754 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:03.490727    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod752af6d4_3cd7_497c_9d85_0a1f6ab46f5f.slice/crio-2461f133fa82e944333bda09c6355283134235fe232ac0f3d26d3b3eacd0c0bd WatchSource:0}: Error finding container 2461f133fa82e944333bda09c6355283134235fe232ac0f3d26d3b3eacd0c0bd: Status 404 returned error can't find the container with id 2461f133fa82e944333bda09c6355283134235fe232ac0f3d26d3b3eacd0c0bd
May 28 08:12:03.584213 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:03.584176    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod21817e48_7f6c_4c5b_bd3c_ada88d6de024.slice/crio-abe19e3de6d2061eb4997bfbe7c87b9b2544d9a08be8bf35045bd864b8504e38 WatchSource:0}: Error finding container abe19e3de6d2061eb4997bfbe7c87b9b2544d9a08be8bf35045bd864b8504e38: Status 404 returned error can't find the container with id abe19e3de6d2061eb4997bfbe7c87b9b2544d9a08be8bf35045bd864b8504e38
May 28 08:12:03.584737 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:03.584709    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18155ab5_6646_4bf6_88af_c74297a72a1e.slice/crio-bf37a3584cba15113a8f2f3f1e833724f6c89e158bac6a6978b28e3f34d44848 WatchSource:0}: Error finding container bf37a3584cba15113a8f2f3f1e833724f6c89e158bac6a6978b28e3f34d44848: Status 404 returned error can't find the container with id bf37a3584cba15113a8f2f3f1e833724f6c89e158bac6a6978b28e3f34d44848
May 28 08:12:03.700099 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.700058    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:03.700266 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.700227    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:03.700325 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.700293    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs podName:90e9cadb-328a-47e1-a119-bae6987ac229 nodeName:}" failed. No retries permitted until 2026-05-28 08:12:04.70027253 +0000 UTC m=+3.136451681 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs") pod "network-metrics-daemon-mlpsc" (UID: "90e9cadb-328a-47e1-a119-bae6987ac229") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:03.801251 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.800546    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jbb2d\" (UniqueName: \"kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d\") pod \"network-check-target-nkgkq\" (UID: \"ecd64268-adf3-40c0-aaaf-510d4c689d8c\") " pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:03.801251 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.800751    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 08:12:03.801251 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.800772    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 08:12:03.801251 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.800787    2577 projected.go:194] Error preparing data for projected volume kube-api-access-jbb2d for pod openshift-network-diagnostics/network-check-target-nkgkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:03.801251 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:03.800846    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d podName:ecd64268-adf3-40c0-aaaf-510d4c689d8c nodeName:}" failed. No retries permitted until 2026-05-28 08:12:04.800827059 +0000 UTC m=+3.237006219 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbb2d" (UniqueName: "kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d") pod "network-check-target-nkgkq" (UID: "ecd64268-adf3-40c0-aaaf-510d4c689d8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:03.934222 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:03.933801    2577 reflector.go:430] "Caches populated" type="*v1.CSIDriver" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 08:12:04.002927 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.002369    2577 reflector.go:430] "Caches populated" type="*v1.Service" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 08:12:04.084712 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.084434    2577 reflector.go:430] "Caches populated" type="*v1.RuntimeClass" reflector="k8s.io/client-go/informers/factory.go:160"
May 28 08:12:04.133470 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.133412    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 08:07:03 +0000 UTC" deadline="2027-11-06 21:09:14.847701053 +0000 UTC"
May 28 08:12:04.133470 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.133454    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="12660h57m10.714251415s"
May 28 08:12:04.247989 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.247926    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8vp6h" event={"ID":"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf","Type":"ContainerStarted","Data":"6cfde6ef64661f6c9f4c5d6cb55e80b6490cf11ae88b5f0b28b7b45849717b4b"}
May 28 08:12:04.265535 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.265435    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bp4g8" event={"ID":"a4b772f7-1b71-4e3b-b2a9-fbf33edfce66","Type":"ContainerStarted","Data":"9b59ed8dd2e03f6f19df6f9ddd874ab400c2938141eb998f0019f35353f443c8"}
May 28 08:12:04.277862 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.277766    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fhw4w" event={"ID":"18155ab5-6646-4bf6-88af-c74297a72a1e","Type":"ContainerStarted","Data":"bf37a3584cba15113a8f2f3f1e833724f6c89e158bac6a6978b28e3f34d44848"}
May 28 08:12:04.281646 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.281581    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch" event={"ID":"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f","Type":"ContainerStarted","Data":"2461f133fa82e944333bda09c6355283134235fe232ac0f3d26d3b3eacd0c0bd"}
May 28 08:12:04.301161 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.301121    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p6k96" event={"ID":"dca0512e-79d3-4c51-9f05-cc6c64fe4f85","Type":"ContainerStarted","Data":"3fd6eedc897c9e740c135e0aa6e12e6d4884814f943179a7c067aeb45511f518"}
May 28 08:12:04.320481 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.320336    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mbdnp" event={"ID":"091802e9-2e2c-4c10-a42c-88623f528cd0","Type":"ContainerStarted","Data":"37fdc0185c48084b2a98cd9c5a163b27dfa7ed45652aac45a3c139366dc59768"}
May 28 08:12:04.328119 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.328080    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal" event={"ID":"47909bb01acf11b78eb7bcc9fa0b9efc","Type":"ContainerStarted","Data":"54c759d2e3f6aceec3beba13d592f3f97af53511047586d6a2f412e17fbff70c"}
May 28 08:12:04.336524 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.336436    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-158.ec2.internal" event={"ID":"ac83f6a6291c1bd1f8450f6fd7149beb","Type":"ContainerStarted","Data":"b7288a074e71a5a2faf941b4fe97d13870cb17c49b1bdb9a230f914c910bafc6"}
May 28 08:12:04.344552 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.344512    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6" event={"ID":"21817e48-7f6c-4c5b-bd3c-ada88d6de024","Type":"ContainerStarted","Data":"abe19e3de6d2061eb4997bfbe7c87b9b2544d9a08be8bf35045bd864b8504e38"}
May 28 08:12:04.350090 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.350053    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j8bt4" event={"ID":"80faa7ab-5aca-4d3f-96cf-f7edc87643fe","Type":"ContainerStarted","Data":"133c024a3d5e8f580200f4fab6a43387adba2708367549d0643a1d499c53da35"}
May 28 08:12:04.709196 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.708542    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:04.709196 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:04.708741    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:04.709196 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:04.708803    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs podName:90e9cadb-328a-47e1-a119-bae6987ac229 nodeName:}" failed. No retries permitted until 2026-05-28 08:12:06.708784405 +0000 UTC m=+5.144963553 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs") pod "network-metrics-daemon-mlpsc" (UID: "90e9cadb-328a-47e1-a119-bae6987ac229") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:04.809289 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:04.809246    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jbb2d\" (UniqueName: \"kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d\") pod \"network-check-target-nkgkq\" (UID: \"ecd64268-adf3-40c0-aaaf-510d4c689d8c\") " pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:04.809502 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:04.809465    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 08:12:04.809502 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:04.809495    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 08:12:04.809601 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:04.809508    2577 projected.go:194] Error preparing data for projected volume kube-api-access-jbb2d for pod openshift-network-diagnostics/network-check-target-nkgkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:04.809601 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:04.809570    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d podName:ecd64268-adf3-40c0-aaaf-510d4c689d8c nodeName:}" failed. No retries permitted until 2026-05-28 08:12:06.809550637 +0000 UTC m=+5.245729796 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbb2d" (UniqueName: "kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d") pod "network-check-target-nkgkq" (UID: "ecd64268-adf3-40c0-aaaf-510d4c689d8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:05.134295 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:05.134058    2577 certificate_manager.go:715] "Certificate rotation deadline determined" logger="kubernetes.io/kubelet-serving" expiration="2028-05-27 08:07:03 +0000 UTC" deadline="2027-12-04 21:45:28.632347569 +0000 UTC"
May 28 08:12:05.134295 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:05.134100    2577 certificate_manager.go:431] "Waiting for next certificate rotation" logger="kubernetes.io/kubelet-serving" sleep="13333h33m23.498251865s"
May 28 08:12:05.237859 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:05.237827    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:05.238048 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:05.237968    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:05.238548 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:05.238394    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:05.238548 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:05.238485    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:06.726622 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:06.726545    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:06.727187 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:06.726754    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:06.727187 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:06.726816    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs podName:90e9cadb-328a-47e1-a119-bae6987ac229 nodeName:}" failed. No retries permitted until 2026-05-28 08:12:10.726797116 +0000 UTC m=+9.162976274 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs") pod "network-metrics-daemon-mlpsc" (UID: "90e9cadb-328a-47e1-a119-bae6987ac229") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:06.827689 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:06.827625    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jbb2d\" (UniqueName: \"kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d\") pod \"network-check-target-nkgkq\" (UID: \"ecd64268-adf3-40c0-aaaf-510d4c689d8c\") " pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:06.827881 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:06.827834    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 08:12:06.827881 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:06.827853    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 08:12:06.827881 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:06.827863    2577 projected.go:194] Error preparing data for projected volume kube-api-access-jbb2d for pod openshift-network-diagnostics/network-check-target-nkgkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:06.828011 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:06.827910    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d podName:ecd64268-adf3-40c0-aaaf-510d4c689d8c nodeName:}" failed. No retries permitted until 2026-05-28 08:12:10.827892791 +0000 UTC m=+9.264071935 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbb2d" (UniqueName: "kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d") pod "network-check-target-nkgkq" (UID: "ecd64268-adf3-40c0-aaaf-510d4c689d8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:07.085246 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.085084    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/node-resolver-k9tpr"]
May 28 08:12:07.103048 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.103011    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k9tpr"
May 28 08:12:07.107038 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.106604    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"openshift-service-ca.crt\""
May 28 08:12:07.107038 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.106761    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"node-resolver-dockercfg-2stwz\""
May 28 08:12:07.107038 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.106867    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"kube-root-ca.crt\""
May 28 08:12:07.130645 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.130613    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-87svz\" (UniqueName: \"kubernetes.io/projected/07b247ee-4fc0-4b9e-ae81-9f260ce894fc-kube-api-access-87svz\") pod \"node-resolver-k9tpr\" (UID: \"07b247ee-4fc0-4b9e-ae81-9f260ce894fc\") " pod="openshift-dns/node-resolver-k9tpr"
May 28 08:12:07.130818 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.130676    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07b247ee-4fc0-4b9e-ae81-9f260ce894fc-tmp-dir\") pod \"node-resolver-k9tpr\" (UID: \"07b247ee-4fc0-4b9e-ae81-9f260ce894fc\") " pod="openshift-dns/node-resolver-k9tpr"
May 28 08:12:07.130818 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.130734    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/07b247ee-4fc0-4b9e-ae81-9f260ce894fc-hosts-file\") pod \"node-resolver-k9tpr\" (UID: \"07b247ee-4fc0-4b9e-ae81-9f260ce894fc\") " pod="openshift-dns/node-resolver-k9tpr"
May 28 08:12:07.231966 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.231917    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/07b247ee-4fc0-4b9e-ae81-9f260ce894fc-hosts-file\") pod \"node-resolver-k9tpr\" (UID: \"07b247ee-4fc0-4b9e-ae81-9f260ce894fc\") " pod="openshift-dns/node-resolver-k9tpr"
May 28 08:12:07.232171 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.232011    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-87svz\" (UniqueName: \"kubernetes.io/projected/07b247ee-4fc0-4b9e-ae81-9f260ce894fc-kube-api-access-87svz\") pod \"node-resolver-k9tpr\" (UID: \"07b247ee-4fc0-4b9e-ae81-9f260ce894fc\") " pod="openshift-dns/node-resolver-k9tpr"
May 28 08:12:07.232171 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.232039    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07b247ee-4fc0-4b9e-ae81-9f260ce894fc-tmp-dir\") pod \"node-resolver-k9tpr\" (UID: \"07b247ee-4fc0-4b9e-ae81-9f260ce894fc\") " pod="openshift-dns/node-resolver-k9tpr"
May 28 08:12:07.232457 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.232431    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/07b247ee-4fc0-4b9e-ae81-9f260ce894fc-tmp-dir\") pod \"node-resolver-k9tpr\" (UID: \"07b247ee-4fc0-4b9e-ae81-9f260ce894fc\") " pod="openshift-dns/node-resolver-k9tpr"
May 28 08:12:07.232546 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.232523    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hosts-file\" (UniqueName: \"kubernetes.io/host-path/07b247ee-4fc0-4b9e-ae81-9f260ce894fc-hosts-file\") pod \"node-resolver-k9tpr\" (UID: \"07b247ee-4fc0-4b9e-ae81-9f260ce894fc\") " pod="openshift-dns/node-resolver-k9tpr"
May 28 08:12:07.238004 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.237977    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:07.238134 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:07.238100    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:07.238550 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.238528    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:07.238689 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:07.238642    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:07.252115 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.252055    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-87svz\" (UniqueName: \"kubernetes.io/projected/07b247ee-4fc0-4b9e-ae81-9f260ce894fc-kube-api-access-87svz\") pod \"node-resolver-k9tpr\" (UID: \"07b247ee-4fc0-4b9e-ae81-9f260ce894fc\") " pod="openshift-dns/node-resolver-k9tpr"
May 28 08:12:07.416669 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:07.416618    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/node-resolver-k9tpr"
May 28 08:12:09.238626 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:09.237736    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:09.238626 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:09.238126    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:09.238626 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:09.237974    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:09.238626 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:09.238588    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:10.760511 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:10.760408    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:10.760972 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:10.760564    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:10.760972 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:10.760635    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs podName:90e9cadb-328a-47e1-a119-bae6987ac229 nodeName:}" failed. No retries permitted until 2026-05-28 08:12:18.760616082 +0000 UTC m=+17.196795242 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs") pod "network-metrics-daemon-mlpsc" (UID: "90e9cadb-328a-47e1-a119-bae6987ac229") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:10.861491 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:10.861457    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jbb2d\" (UniqueName: \"kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d\") pod \"network-check-target-nkgkq\" (UID: \"ecd64268-adf3-40c0-aaaf-510d4c689d8c\") " pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:10.861687 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:10.861649    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 08:12:10.861753 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:10.861696    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 08:12:10.861753 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:10.861711    2577 projected.go:194] Error preparing data for projected volume kube-api-access-jbb2d for pod openshift-network-diagnostics/network-check-target-nkgkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:10.861844 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:10.861775    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d podName:ecd64268-adf3-40c0-aaaf-510d4c689d8c nodeName:}" failed. No retries permitted until 2026-05-28 08:12:18.86175261 +0000 UTC m=+17.297931767 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbb2d" (UniqueName: "kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d") pod "network-check-target-nkgkq" (UID: "ecd64268-adf3-40c0-aaaf-510d4c689d8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:11.237977 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:11.237899    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:11.238144 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:11.237899    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:11.238144 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:11.238051    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:11.238144 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:11.238094    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:13.237406 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:13.237368    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:13.237873 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:13.237372    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:13.237873 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:13.237503    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:13.237873 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:13.237590    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:15.237854 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:15.237819    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:15.238266 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:15.237828    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:15.238266 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:15.237940    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:15.238266 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:15.238037    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:17.237091 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:17.237040    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:17.237512 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:17.237048    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:17.237512 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:17.237173    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:17.237512 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:17.237284    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:18.817437 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:18.817396    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:18.817949 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:18.817586    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:18.817949 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:18.817681    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs podName:90e9cadb-328a-47e1-a119-bae6987ac229 nodeName:}" failed. No retries permitted until 2026-05-28 08:12:34.817641522 +0000 UTC m=+33.253820670 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs") pod "network-metrics-daemon-mlpsc" (UID: "90e9cadb-328a-47e1-a119-bae6987ac229") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:18.918277 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:18.918240    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jbb2d\" (UniqueName: \"kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d\") pod \"network-check-target-nkgkq\" (UID: \"ecd64268-adf3-40c0-aaaf-510d4c689d8c\") " pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:18.918470 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:18.918414    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 08:12:18.918470 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:18.918435    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 08:12:18.918470 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:18.918448    2577 projected.go:194] Error preparing data for projected volume kube-api-access-jbb2d for pod openshift-network-diagnostics/network-check-target-nkgkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:18.918632 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:18.918523    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d podName:ecd64268-adf3-40c0-aaaf-510d4c689d8c nodeName:}" failed. No retries permitted until 2026-05-28 08:12:34.91850417 +0000 UTC m=+33.354683331 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbb2d" (UniqueName: "kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d") pod "network-check-target-nkgkq" (UID: "ecd64268-adf3-40c0-aaaf-510d4c689d8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:19.237360 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:19.237276    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:19.237527 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:19.237278    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:19.237527 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:19.237395    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:19.237527 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:19.237483    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:21.237582 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:21.237556    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:21.238133 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:21.237556    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:21.238133 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:21.237690    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:21.238133 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:21.237752    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:22.264357 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:12:22.264325    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod07b247ee_4fc0_4b9e_ae81_9f260ce894fc.slice/crio-c9b84d0ccbddc390c133ba7366ea3473a50e89b0fd96741e309ec23eb7858919 WatchSource:0}: Error finding container c9b84d0ccbddc390c133ba7366ea3473a50e89b0fd96741e309ec23eb7858919: Status 404 returned error can't find the container with id c9b84d0ccbddc390c133ba7366ea3473a50e89b0fd96741e309ec23eb7858919
May 28 08:12:22.381263 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:22.381235    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k9tpr" event={"ID":"07b247ee-4fc0-4b9e-ae81-9f260ce894fc","Type":"ContainerStarted","Data":"c9b84d0ccbddc390c133ba7366ea3473a50e89b0fd96741e309ec23eb7858919"}
May 28 08:12:23.237541 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.237243    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:23.237647 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.237258    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:23.237743 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:23.237647    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:23.237779 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:23.237755    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:23.385443 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.385324    2577 generic.go:358] "Generic (PLEG): container finished" podID="48fe84ba-9a15-4a3d-a449-fb2694fbdfcf" containerID="06590bab13609ff979c36e1798671c6bb809da05073fa42a2808bbde0992cdaa" exitCode=0
May 28 08:12:23.385443 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.385416    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8vp6h" event={"ID":"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf","Type":"ContainerDied","Data":"06590bab13609ff979c36e1798671c6bb809da05073fa42a2808bbde0992cdaa"}
May 28 08:12:23.386886 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.386855    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/node-ca-bp4g8" event={"ID":"a4b772f7-1b71-4e3b-b2a9-fbf33edfce66","Type":"ContainerStarted","Data":"3df1e408671dfb34acb8597b27a41c88fde86a8c0b11f89620846bdb2daff6f0"}
May 28 08:12:23.388231 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.388195    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-node-tuning-operator/tuned-fhw4w" event={"ID":"18155ab5-6646-4bf6-88af-c74297a72a1e","Type":"ContainerStarted","Data":"ee8b2f59d383ac61124a3872094973e02b360e0718a3e0abbb893bb8b666ef59"}
May 28 08:12:23.389593 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.389574    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch" event={"ID":"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f","Type":"ContainerStarted","Data":"14656b7122209d78a23b906634cbf5b813e0762c04f9cda9d57e2480cfbc6ef9"}
May 28 08:12:23.394430 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.394405    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-mbdnp" event={"ID":"091802e9-2e2c-4c10-a42c-88623f528cd0","Type":"ContainerStarted","Data":"37d05972bd401673809d889ad6ddc2b8f53649db7ef92879a6d63d94d03ebe27"}
May 28 08:12:23.396065 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.396043    2577 generic.go:358] "Generic (PLEG): container finished" podID="47909bb01acf11b78eb7bcc9fa0b9efc" containerID="38ace7a5c441fa8b950904d9e7d56ce9c0f459a69d021c7754fb838e68caa50f" exitCode=0
May 28 08:12:23.396160 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.396113    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal" event={"ID":"47909bb01acf11b78eb7bcc9fa0b9efc","Type":"ContainerDied","Data":"38ace7a5c441fa8b950904d9e7d56ce9c0f459a69d021c7754fb838e68caa50f"}
May 28 08:12:23.397475 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.397449    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-158.ec2.internal" event={"ID":"ac83f6a6291c1bd1f8450f6fd7149beb","Type":"ContainerStarted","Data":"b3506bcf089bbe11cf80f699e19d93160299a6ae5b503a1acad2b59a38445cec"}
May 28 08:12:23.398947 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.398922    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/node-resolver-k9tpr" event={"ID":"07b247ee-4fc0-4b9e-ae81-9f260ce894fc","Type":"ContainerStarted","Data":"10639275f2c96a1632929f81b5f443863076f23ffc56376073cdfa2fcc531de6"}
May 28 08:12:23.401776 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.401751    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6" event={"ID":"21817e48-7f6c-4c5b-bd3c-ada88d6de024","Type":"ContainerStarted","Data":"7b369dd5229e939a333f3d00583816ce62f67055d506c0055535d5061b4a8788"}
May 28 08:12:23.401881 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.401784    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6" event={"ID":"21817e48-7f6c-4c5b-bd3c-ada88d6de024","Type":"ContainerStarted","Data":"efc88cb1452d41f4aaa76613cdefe286eb12cbd094f251204f7de64e5624a9ce"}
May 28 08:12:23.401881 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.401797    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6" event={"ID":"21817e48-7f6c-4c5b-bd3c-ada88d6de024","Type":"ContainerStarted","Data":"57c191fb6791cc178025a81a556102ae4c0d4a928845365310fa7ab31b900831"}
May 28 08:12:23.401881 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.401809    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6" event={"ID":"21817e48-7f6c-4c5b-bd3c-ada88d6de024","Type":"ContainerStarted","Data":"c20e82a471e3655637c4d6d01cfafb31c231c5e7505e426e78a722d635f517c7"}
May 28 08:12:23.401881 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.401822    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6" event={"ID":"21817e48-7f6c-4c5b-bd3c-ada88d6de024","Type":"ContainerStarted","Data":"2404aa268cf97b55c46da0e7e872bfa2921322389b799f4aba727e5273495bb3"}
May 28 08:12:23.401881 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.401834    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6" event={"ID":"21817e48-7f6c-4c5b-bd3c-ada88d6de024","Type":"ContainerStarted","Data":"0b6c19a04a9e112928e55c38513e5a9ed04ff75450eff8de1cb408f1a20631d3"}
May 28 08:12:23.403101 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.403079    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/konnectivity-agent-j8bt4" event={"ID":"80faa7ab-5aca-4d3f-96cf-f7edc87643fe","Type":"ContainerStarted","Data":"21c9df8418e644e2aaa682c8ea706e57086e204150737ec3069790f30ea3e04a"}
May 28 08:12:23.428092 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.428033    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/kube-apiserver-proxy-ip-10-0-137-158.ec2.internal" podStartSLOduration=21.428013707 podStartE2EDuration="21.428013707s" podCreationTimestamp="2026-05-28 08:12:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:12:23.426047235 +0000 UTC m=+21.862226403" watchObservedRunningTime="2026-05-28 08:12:23.428013707 +0000 UTC m=+21.864192874"
May 28 08:12:23.463832 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.463782    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-mbdnp" podStartSLOduration=2.601588391 podStartE2EDuration="21.463769255s" podCreationTimestamp="2026-05-28 08:12:02 +0000 UTC" firstStartedPulling="2026-05-28 08:12:03.43542795 +0000 UTC m=+1.871607096" lastFinishedPulling="2026-05-28 08:12:22.297608808 +0000 UTC m=+20.733787960" observedRunningTime="2026-05-28 08:12:23.463363938 +0000 UTC m=+21.899543104" watchObservedRunningTime="2026-05-28 08:12:23.463769255 +0000 UTC m=+21.899948421"
May 28 08:12:23.479816 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.479712    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/konnectivity-agent-j8bt4" podStartSLOduration=2.693732138 podStartE2EDuration="21.479692963s" podCreationTimestamp="2026-05-28 08:12:02 +0000 UTC" firstStartedPulling="2026-05-28 08:12:03.474549759 +0000 UTC m=+1.910728903" lastFinishedPulling="2026-05-28 08:12:22.260510584 +0000 UTC m=+20.696689728" observedRunningTime="2026-05-28 08:12:23.478678903 +0000 UTC m=+21.914858071" watchObservedRunningTime="2026-05-28 08:12:23.479692963 +0000 UTC m=+21.915872130"
May 28 08:12:23.495945 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.495896    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-node-tuning-operator/tuned-fhw4w" podStartSLOduration=2.821805942 podStartE2EDuration="21.495879028s" podCreationTimestamp="2026-05-28 08:12:02 +0000 UTC" firstStartedPulling="2026-05-28 08:12:03.586427433 +0000 UTC m=+2.022606577" lastFinishedPulling="2026-05-28 08:12:22.260500502 +0000 UTC m=+20.696679663" observedRunningTime="2026-05-28 08:12:23.495789292 +0000 UTC m=+21.931968460" watchObservedRunningTime="2026-05-28 08:12:23.495879028 +0000 UTC m=+21.932058194"
May 28 08:12:23.514141 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.514092    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/node-ca-bp4g8" podStartSLOduration=2.663185878 podStartE2EDuration="21.514078266s" podCreationTimestamp="2026-05-28 08:12:02 +0000 UTC" firstStartedPulling="2026-05-28 08:12:03.410055133 +0000 UTC m=+1.846234277" lastFinishedPulling="2026-05-28 08:12:22.260947507 +0000 UTC m=+20.697126665" observedRunningTime="2026-05-28 08:12:23.513730437 +0000 UTC m=+21.949909603" watchObservedRunningTime="2026-05-28 08:12:23.514078266 +0000 UTC m=+21.950257432"
May 28 08:12:23.754964 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.754887    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="kube-system/konnectivity-agent-j8bt4"
May 28 08:12:23.755576 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.755555    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="kube-system/konnectivity-agent-j8bt4"
May 28 08:12:23.785313 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.785273    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/node-resolver-k9tpr" podStartSLOduration=16.785254562 podStartE2EDuration="16.785254562s" podCreationTimestamp="2026-05-28 08:12:07 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:12:23.529334273 +0000 UTC m=+21.965513458" watchObservedRunningTime="2026-05-28 08:12:23.785254562 +0000 UTC m=+22.221433729"
May 28 08:12:23.893697 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:23.893675    2577 plugin_watcher.go:194] "Adding socket path or updating timestamp to desired state cache" path="/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock"
May 28 08:12:24.166206 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:24.166109    2577 reconciler.go:161] "OperationExecutor.RegisterPlugin started" plugin={"SocketPath":"/var/lib/kubelet/plugins_registry/ebs.csi.aws.com-reg.sock","Timestamp":"2026-05-28T08:12:23.893692625Z","UUID":"71649be9-c357-44a8-938d-d65de697a34a","Handler":null,"Name":"","Endpoint":""}
May 28 08:12:24.169294 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:24.169265    2577 csi_plugin.go:106] kubernetes.io/csi: Trying to validate a new CSI Driver with name: ebs.csi.aws.com endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock versions: 1.0.0
May 28 08:12:24.169294 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:24.169298    2577 csi_plugin.go:119] kubernetes.io/csi: Register new plugin with name: ebs.csi.aws.com at endpoint: /var/lib/kubelet/plugins/ebs.csi.aws.com/csi.sock
May 28 08:12:24.407631 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:24.407551    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch" event={"ID":"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f","Type":"ContainerStarted","Data":"4c92984162451130ec9eb6ed4f5ec59f4321181c8a4ad99385df168d15bf9975"}
May 28 08:12:24.409170 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:24.409134    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-operator/iptables-alerter-p6k96" event={"ID":"dca0512e-79d3-4c51-9f05-cc6c64fe4f85","Type":"ContainerStarted","Data":"de18074b70a1327dd0ed83e3ac6d2a1e280655db6d71a626b294c074136d2cfd"}
May 28 08:12:24.413482 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:24.413455    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal" event={"ID":"47909bb01acf11b78eb7bcc9fa0b9efc","Type":"ContainerStarted","Data":"feaf71926db31cfae4e4f5c56c18878ded732455c4eb55e507bd0318b40a61dd"}
May 28 08:12:24.415351 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:24.415317    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="kube-system/konnectivity-agent-j8bt4"
May 28 08:12:24.415475 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:24.415442    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="kube-system/konnectivity-agent-j8bt4"
May 28 08:12:24.425166 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:24.425117    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-operator/iptables-alerter-p6k96" podStartSLOduration=3.622394488 podStartE2EDuration="22.425102504s" podCreationTimestamp="2026-05-28 08:12:02 +0000 UTC" firstStartedPulling="2026-05-28 08:12:03.459227144 +0000 UTC m=+1.895406288" lastFinishedPulling="2026-05-28 08:12:22.261935145 +0000 UTC m=+20.698114304" observedRunningTime="2026-05-28 08:12:24.424516848 +0000 UTC m=+22.860696016" watchObservedRunningTime="2026-05-28 08:12:24.425102504 +0000 UTC m=+22.861281672"
May 28 08:12:24.438851 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:24.438797    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-machine-config-operator/kube-rbac-proxy-crio-ip-10-0-137-158.ec2.internal" podStartSLOduration=22.438780025 podStartE2EDuration="22.438780025s" podCreationTimestamp="2026-05-28 08:12:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:12:24.438562842 +0000 UTC m=+22.874742008" watchObservedRunningTime="2026-05-28 08:12:24.438780025 +0000 UTC m=+22.874959192"
May 28 08:12:25.237283 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:25.237259    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:25.237396 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:25.237257    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:25.237396 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:25.237367    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:25.237501 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:25.237451    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:25.417030 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:25.416989    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch" event={"ID":"752af6d4-3cd7-497c-9d85-0a1f6ab46f5f","Type":"ContainerStarted","Data":"07e5adbe968cd1bca5cd104eadecd6183c78441e293439fbea2ff099a72c4452"}
May 28 08:12:25.419615 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:25.419587    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6" event={"ID":"21817e48-7f6c-4c5b-bd3c-ada88d6de024","Type":"ContainerStarted","Data":"1995a161e5e84a525779c716ae513c80415ad6248fe4dd09ccfa53141f347b54"}
May 28 08:12:25.436866 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:25.436818    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-cluster-csi-drivers/aws-ebs-csi-driver-node-4v4ch" podStartSLOduration=2.218122642 podStartE2EDuration="23.436805192s" podCreationTimestamp="2026-05-28 08:12:02 +0000 UTC" firstStartedPulling="2026-05-28 08:12:03.492390369 +0000 UTC m=+1.928569526" lastFinishedPulling="2026-05-28 08:12:24.711072928 +0000 UTC m=+23.147252076" observedRunningTime="2026-05-28 08:12:25.436444652 +0000 UTC m=+23.872623817" watchObservedRunningTime="2026-05-28 08:12:25.436805192 +0000 UTC m=+23.872984357"
May 28 08:12:27.237906 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:27.237797    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:27.238286 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:27.237811    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:27.238286 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:27.237915    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:27.238286 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:27.238021    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:28.431024 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:28.430503    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6" event={"ID":"21817e48-7f6c-4c5b-bd3c-ada88d6de024","Type":"ContainerStarted","Data":"0e619a6856bec8864c98bc86058d22c230fc5712d0fa6ee3b50d1c3c1dab95e3"}
May 28 08:12:28.431024 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:28.430833    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:28.431024 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:28.430855    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:28.447345 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:28.447323    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:28.460252 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:28.460159    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6" podStartSLOduration=7.446825367 podStartE2EDuration="26.460141836s" podCreationTimestamp="2026-05-28 08:12:02 +0000 UTC" firstStartedPulling="2026-05-28 08:12:03.585907258 +0000 UTC m=+2.022086405" lastFinishedPulling="2026-05-28 08:12:22.59922373 +0000 UTC m=+21.035402874" observedRunningTime="2026-05-28 08:12:28.459648366 +0000 UTC m=+26.895827532" watchObservedRunningTime="2026-05-28 08:12:28.460141836 +0000 UTC m=+26.896321003"
May 28 08:12:29.237444 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:29.237409    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:29.237641 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:29.237521    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:29.237641 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:29.237557    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:29.237641 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:29.237615    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:29.434109 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:29.434079    2577 generic.go:358] "Generic (PLEG): container finished" podID="48fe84ba-9a15-4a3d-a449-fb2694fbdfcf" containerID="2d0d0436ded0c7bb007f3a530c2d49ed48fe662e377bea029b1fcd91a3ce0d02" exitCode=0
May 28 08:12:29.435055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:29.434164    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8vp6h" event={"ID":"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf","Type":"ContainerDied","Data":"2d0d0436ded0c7bb007f3a530c2d49ed48fe662e377bea029b1fcd91a3ce0d02"}
May 28 08:12:29.435055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:29.434866    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:29.450982 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:29.450958    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:12:30.327348 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:30.327032    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nkgkq"]
May 28 08:12:30.327493 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:30.327373    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:30.327542 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:30.327485    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:30.327845 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:30.327823    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mlpsc"]
May 28 08:12:30.327956 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:30.327944    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:30.328078 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:30.328057    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:30.437494 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:30.437461    2577 generic.go:358] "Generic (PLEG): container finished" podID="48fe84ba-9a15-4a3d-a449-fb2694fbdfcf" containerID="91547b3a05f59a8dee464f6c7e8d4ba8bad81e601d6b51661980ed07c3171b51" exitCode=0
May 28 08:12:30.437880 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:30.437543    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8vp6h" event={"ID":"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf","Type":"ContainerDied","Data":"91547b3a05f59a8dee464f6c7e8d4ba8bad81e601d6b51661980ed07c3171b51"}
May 28 08:12:31.441375 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:31.441331    2577 generic.go:358] "Generic (PLEG): container finished" podID="48fe84ba-9a15-4a3d-a449-fb2694fbdfcf" containerID="4532f0fe43890bcd9ad292ad75a822dc16ca3e469bdb75b79ee942a2cb76e2a8" exitCode=0
May 28 08:12:31.441769 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:31.441416    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8vp6h" event={"ID":"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf","Type":"ContainerDied","Data":"4532f0fe43890bcd9ad292ad75a822dc16ca3e469bdb75b79ee942a2cb76e2a8"}
May 28 08:12:32.238055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:32.237968    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:32.238223 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:32.238077    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:32.238223 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:32.238166    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:32.238329 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:32.238273    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:34.237852 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:34.237766    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:34.238293 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:34.237908    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-network-diagnostics/network-check-target-nkgkq" podUID="ecd64268-adf3-40c0-aaaf-510d4c689d8c"
May 28 08:12:34.238293 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:34.237939    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:34.238293 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:34.238011    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="network is not ready: container runtime network not ready: NetworkReady=false reason:NetworkPluginNotReady message:Network plugin returns error: no CNI configuration file in /etc/kubernetes/cni/net.d/. Has your network provider started?" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:12:34.828789 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:34.828743    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:34.828982 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:34.828883    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:34.828982 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:34.828954    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs podName:90e9cadb-328a-47e1-a119-bae6987ac229 nodeName:}" failed. No retries permitted until 2026-05-28 08:13:06.828934873 +0000 UTC m=+65.265114020 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs") pod "network-metrics-daemon-mlpsc" (UID: "90e9cadb-328a-47e1-a119-bae6987ac229") : object "openshift-multus"/"metrics-daemon-secret" not registered
May 28 08:12:34.929908 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:34.929872    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jbb2d\" (UniqueName: \"kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d\") pod \"network-check-target-nkgkq\" (UID: \"ecd64268-adf3-40c0-aaaf-510d4c689d8c\") " pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:34.930087 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:34.930013    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/kube-root-ca.crt: object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered
May 28 08:12:34.930087 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:34.930027    2577 projected.go:289] Couldn't get configMap openshift-network-diagnostics/openshift-service-ca.crt: object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered
May 28 08:12:34.930087 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:34.930036    2577 projected.go:194] Error preparing data for projected volume kube-api-access-jbb2d for pod openshift-network-diagnostics/network-check-target-nkgkq: [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:34.930222 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:34.930095    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d podName:ecd64268-adf3-40c0-aaaf-510d4c689d8c nodeName:}" failed. No retries permitted until 2026-05-28 08:13:06.930081676 +0000 UTC m=+65.366260836 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "kube-api-access-jbb2d" (UniqueName: "kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d") pod "network-check-target-nkgkq" (UID: "ecd64268-adf3-40c0-aaaf-510d4c689d8c") : [object "openshift-network-diagnostics"/"kube-root-ca.crt" not registered, object "openshift-network-diagnostics"/"openshift-service-ca.crt" not registered]
May 28 08:12:35.342411 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.342383    2577 kubelet_node_status.go:736] "Recording event message for node" node="ip-10-0-137-158.ec2.internal" event="NodeReady"
May 28 08:12:35.342888 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.342551    2577 kubelet_node_status.go:550] "Fast updating node status as it just became ready"
May 28 08:12:35.386178 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.386139    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-dns/dns-default-vcwq4"]
May 28 08:12:35.410421 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.410389    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress-canary/ingress-canary-sjzdf"]
May 28 08:12:35.426433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.426398    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vcwq4"]
May 28 08:12:35.426433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.426437    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sjzdf"]
May 28 08:12:35.426688 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.426536    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:12:35.426688 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.426553    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:35.429456 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.429437    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-dns\"/\"dns-default\""
May 28 08:12:35.429739 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.429722    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"canary-serving-cert\""
May 28 08:12:35.429820 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.429776    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-22w5z\""
May 28 08:12:35.430003 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.429984    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-default-metrics-tls\""
May 28 08:12:35.430003 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.429990    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"kube-root-ca.crt\""
May 28 08:12:35.430141 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.430031    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress-canary\"/\"openshift-service-ca.crt\""
May 28 08:12:35.430141 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.429997    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cs8td\""
May 28 08:12:35.535127 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.535081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rfpgx\" (UniqueName: \"kubernetes.io/projected/50981871-e522-44dc-b0a5-209d7ec9958f-kube-api-access-rfpgx\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:12:35.535336 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.535153    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/48d7931c-9e47-424d-aaaf-1d3c90add99a-config-volume\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:35.535336 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.535222    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bkl2f\" (UniqueName: \"kubernetes.io/projected/48d7931c-9e47-424d-aaaf-1d3c90add99a-kube-api-access-bkl2f\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:35.535336 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.535257    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48d7931c-9e47-424d-aaaf-1d3c90add99a-tmp-dir\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:35.535488 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.535337    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:12:35.535488 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.535374    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:35.636184 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.636101    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/48d7931c-9e47-424d-aaaf-1d3c90add99a-config-volume\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:35.636184 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.636163    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bkl2f\" (UniqueName: \"kubernetes.io/projected/48d7931c-9e47-424d-aaaf-1d3c90add99a-kube-api-access-bkl2f\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:35.636420 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.636196    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48d7931c-9e47-424d-aaaf-1d3c90add99a-tmp-dir\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:35.636420 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.636236    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:12:35.636420 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.636258    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:35.636420 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:35.636378    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 08:12:35.636420 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:35.636411    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 08:12:35.636766 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:35.636439    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls podName:48d7931c-9e47-424d-aaaf-1d3c90add99a nodeName:}" failed. No retries permitted until 2026-05-28 08:12:36.136419976 +0000 UTC m=+34.572599120 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls") pod "dns-default-vcwq4" (UID: "48d7931c-9e47-424d-aaaf-1d3c90add99a") : secret "dns-default-metrics-tls" not found
May 28 08:12:35.636766 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:35.636481    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert podName:50981871-e522-44dc-b0a5-209d7ec9958f nodeName:}" failed. No retries permitted until 2026-05-28 08:12:36.136464483 +0000 UTC m=+34.572643634 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert") pod "ingress-canary-sjzdf" (UID: "50981871-e522-44dc-b0a5-209d7ec9958f") : secret "canary-serving-cert" not found
May 28 08:12:35.636766 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.636545    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rfpgx\" (UniqueName: \"kubernetes.io/projected/50981871-e522-44dc-b0a5-209d7ec9958f-kube-api-access-rfpgx\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:12:35.636766 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.636620    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tmp-dir\" (UniqueName: \"kubernetes.io/empty-dir/48d7931c-9e47-424d-aaaf-1d3c90add99a-tmp-dir\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:35.636931 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.636820    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/configmap/48d7931c-9e47-424d-aaaf-1d3c90add99a-config-volume\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:35.647621 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.647499    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bkl2f\" (UniqueName: \"kubernetes.io/projected/48d7931c-9e47-424d-aaaf-1d3c90add99a-kube-api-access-bkl2f\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:35.647775 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:35.647682    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rfpgx\" (UniqueName: \"kubernetes.io/projected/50981871-e522-44dc-b0a5-209d7ec9958f-kube-api-access-rfpgx\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:12:36.140694 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:36.140639    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:12:36.140694 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:36.140702    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:36.140987 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:36.140807    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 08:12:36.140987 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:36.140839    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 08:12:36.140987 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:36.140898    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls podName:48d7931c-9e47-424d-aaaf-1d3c90add99a nodeName:}" failed. No retries permitted until 2026-05-28 08:12:37.140877188 +0000 UTC m=+35.577056332 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls") pod "dns-default-vcwq4" (UID: "48d7931c-9e47-424d-aaaf-1d3c90add99a") : secret "dns-default-metrics-tls" not found
May 28 08:12:36.140987 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:36.140915    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert podName:50981871-e522-44dc-b0a5-209d7ec9958f nodeName:}" failed. No retries permitted until 2026-05-28 08:12:37.140907044 +0000 UTC m=+35.577086187 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert") pod "ingress-canary-sjzdf" (UID: "50981871-e522-44dc-b0a5-209d7ec9958f") : secret "canary-serving-cert" not found
May 28 08:12:36.238499 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:36.238034    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:12:36.238499 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:36.238073    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:12:36.241763 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:36.241734    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 08:12:36.242065 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:36.242048    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 08:12:36.242555 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:36.242529    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 08:12:36.242684 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:36.242532    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s6dkv\""
May 28 08:12:36.242684 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:36.242538    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-flvgj\""
May 28 08:12:37.147073 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:37.147036    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:12:37.147073 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:37.147075    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:37.147603 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:37.147180    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 08:12:37.147603 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:37.147202    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 08:12:37.147603 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:37.147236    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls podName:48d7931c-9e47-424d-aaaf-1d3c90add99a nodeName:}" failed. No retries permitted until 2026-05-28 08:12:39.147220897 +0000 UTC m=+37.583400041 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls") pod "dns-default-vcwq4" (UID: "48d7931c-9e47-424d-aaaf-1d3c90add99a") : secret "dns-default-metrics-tls" not found
May 28 08:12:37.147603 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:37.147269    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert podName:50981871-e522-44dc-b0a5-209d7ec9958f nodeName:}" failed. No retries permitted until 2026-05-28 08:12:39.147251937 +0000 UTC m=+37.583431094 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert") pod "ingress-canary-sjzdf" (UID: "50981871-e522-44dc-b0a5-209d7ec9958f") : secret "canary-serving-cert" not found
May 28 08:12:38.457674 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:38.457627    2577 generic.go:358] "Generic (PLEG): container finished" podID="48fe84ba-9a15-4a3d-a449-fb2694fbdfcf" containerID="0a163201827c61f16db19bf06297534c862fd3db7f47756f3d75db066cdc399b" exitCode=0
May 28 08:12:38.458096 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:38.457714    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8vp6h" event={"ID":"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf","Type":"ContainerDied","Data":"0a163201827c61f16db19bf06297534c862fd3db7f47756f3d75db066cdc399b"}
May 28 08:12:39.162107 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:39.162070    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:12:39.162107 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:39.162109    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:39.162303 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:39.162237    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 08:12:39.162303 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:39.162241    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 08:12:39.162370 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:39.162308    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls podName:48d7931c-9e47-424d-aaaf-1d3c90add99a nodeName:}" failed. No retries permitted until 2026-05-28 08:12:43.16228963 +0000 UTC m=+41.598468774 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls") pod "dns-default-vcwq4" (UID: "48d7931c-9e47-424d-aaaf-1d3c90add99a") : secret "dns-default-metrics-tls" not found
May 28 08:12:39.162370 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:39.162322    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert podName:50981871-e522-44dc-b0a5-209d7ec9958f nodeName:}" failed. No retries permitted until 2026-05-28 08:12:43.162316108 +0000 UTC m=+41.598495252 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert") pod "ingress-canary-sjzdf" (UID: "50981871-e522-44dc-b0a5-209d7ec9958f") : secret "canary-serving-cert" not found
May 28 08:12:39.462526 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:39.462445    2577 generic.go:358] "Generic (PLEG): container finished" podID="48fe84ba-9a15-4a3d-a449-fb2694fbdfcf" containerID="f4c877bd15826fa852e0fd2ddc86161365c597290aa4c7ac5b82216960e1fc13" exitCode=0
May 28 08:12:39.462526 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:39.462498    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8vp6h" event={"ID":"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf","Type":"ContainerDied","Data":"f4c877bd15826fa852e0fd2ddc86161365c597290aa4c7ac5b82216960e1fc13"}
May 28 08:12:40.467222 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:40.467181    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/multus-additional-cni-plugins-8vp6h" event={"ID":"48fe84ba-9a15-4a3d-a449-fb2694fbdfcf","Type":"ContainerStarted","Data":"ac74ec2dc1473005d1eef9b7763d10f95e9377d0609d5501137a69f319cb2921"}
May 28 08:12:40.490556 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:40.490495    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/multus-additional-cni-plugins-8vp6h" podStartSLOduration=4.482774779 podStartE2EDuration="38.490480325s" podCreationTimestamp="2026-05-28 08:12:02 +0000 UTC" firstStartedPulling="2026-05-28 08:12:03.424264086 +0000 UTC m=+1.860443230" lastFinishedPulling="2026-05-28 08:12:37.431969614 +0000 UTC m=+35.868148776" observedRunningTime="2026-05-28 08:12:40.490168936 +0000 UTC m=+38.926348120" watchObservedRunningTime="2026-05-28 08:12:40.490480325 +0000 UTC m=+38.926659491"
May 28 08:12:43.192168 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:43.192130    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:12:43.192168 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:43.192170    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:43.192590 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:43.192294    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 08:12:43.192590 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:43.192303    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 08:12:43.192590 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:43.192384    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert podName:50981871-e522-44dc-b0a5-209d7ec9958f nodeName:}" failed. No retries permitted until 2026-05-28 08:12:51.192365936 +0000 UTC m=+49.628545093 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert") pod "ingress-canary-sjzdf" (UID: "50981871-e522-44dc-b0a5-209d7ec9958f") : secret "canary-serving-cert" not found
May 28 08:12:43.192590 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:43.192399    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls podName:48d7931c-9e47-424d-aaaf-1d3c90add99a nodeName:}" failed. No retries permitted until 2026-05-28 08:12:51.192393588 +0000 UTC m=+49.628572732 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls") pod "dns-default-vcwq4" (UID: "48d7931c-9e47-424d-aaaf-1d3c90add99a") : secret "dns-default-metrics-tls" not found
May 28 08:12:51.242181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:51.242143    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:12:51.242181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:12:51.242183    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:12:51.242702 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:51.242299    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 08:12:51.242702 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:51.242299    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 08:12:51.242702 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:51.242366    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls podName:48d7931c-9e47-424d-aaaf-1d3c90add99a nodeName:}" failed. No retries permitted until 2026-05-28 08:13:07.242344553 +0000 UTC m=+65.678523710 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls") pod "dns-default-vcwq4" (UID: "48d7931c-9e47-424d-aaaf-1d3c90add99a") : secret "dns-default-metrics-tls" not found
May 28 08:12:51.242702 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:12:51.242382    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert podName:50981871-e522-44dc-b0a5-209d7ec9958f nodeName:}" failed. No retries permitted until 2026-05-28 08:13:07.242373561 +0000 UTC m=+65.678552706 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert") pod "ingress-canary-sjzdf" (UID: "50981871-e522-44dc-b0a5-209d7ec9958f") : secret "canary-serving-cert" not found
May 28 08:13:01.452725 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:01.452696    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ovn-kubernetes/ovnkube-node-hpqw6"
May 28 08:13:06.856560 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:06.856520    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:13:06.859947 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:06.859926    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-secret\""
May 28 08:13:06.867582 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:13:06.867559    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 08:13:06.867649 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:13:06.867632    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs podName:90e9cadb-328a-47e1-a119-bae6987ac229 nodeName:}" failed. No retries permitted until 2026-05-28 08:14:10.867615442 +0000 UTC m=+129.303794590 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs") pod "network-metrics-daemon-mlpsc" (UID: "90e9cadb-328a-47e1-a119-bae6987ac229") : secret "metrics-daemon-secret" not found
May 28 08:13:06.956865 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:06.956821    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jbb2d\" (UniqueName: \"kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d\") pod \"network-check-target-nkgkq\" (UID: \"ecd64268-adf3-40c0-aaaf-510d4c689d8c\") " pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:13:06.960559 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:06.960541    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"kube-root-ca.crt\""
May 28 08:13:06.970642 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:06.970621    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-network-diagnostics\"/\"openshift-service-ca.crt\""
May 28 08:13:06.981042 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:06.981021    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jbb2d\" (UniqueName: \"kubernetes.io/projected/ecd64268-adf3-40c0-aaaf-510d4c689d8c-kube-api-access-jbb2d\") pod \"network-check-target-nkgkq\" (UID: \"ecd64268-adf3-40c0-aaaf-510d4c689d8c\") " pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:13:07.155047 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:07.154964    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-network-diagnostics\"/\"default-dockercfg-flvgj\""
May 28 08:13:07.162623 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:07.162604    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:13:07.259013 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:07.258980    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:13:07.259159 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:07.259024    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:13:07.259159 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:13:07.259135    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 08:13:07.259159 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:13:07.259139    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 08:13:07.259316 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:13:07.259199    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls podName:48d7931c-9e47-424d-aaaf-1d3c90add99a nodeName:}" failed. No retries permitted until 2026-05-28 08:13:39.259181392 +0000 UTC m=+97.695360555 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls") pod "dns-default-vcwq4" (UID: "48d7931c-9e47-424d-aaaf-1d3c90add99a") : secret "dns-default-metrics-tls" not found
May 28 08:13:07.259316 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:13:07.259216    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert podName:50981871-e522-44dc-b0a5-209d7ec9958f nodeName:}" failed. No retries permitted until 2026-05-28 08:13:39.259208359 +0000 UTC m=+97.695387502 (durationBeforeRetry 32s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert") pod "ingress-canary-sjzdf" (UID: "50981871-e522-44dc-b0a5-209d7ec9958f") : secret "canary-serving-cert" not found
May 28 08:13:07.355893 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:07.355851    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-network-diagnostics/network-check-target-nkgkq"]
May 28 08:13:07.358994 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:13:07.358965    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podecd64268_adf3_40c0_aaaf_510d4c689d8c.slice/crio-030cb7a19a3e8014b276c8726809943eb06ea38e110cbb408f03461d0c89c82d WatchSource:0}: Error finding container 030cb7a19a3e8014b276c8726809943eb06ea38e110cbb408f03461d0c89c82d: Status 404 returned error can't find the container with id 030cb7a19a3e8014b276c8726809943eb06ea38e110cbb408f03461d0c89c82d
May 28 08:13:07.516890 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:07.516829    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nkgkq" event={"ID":"ecd64268-adf3-40c0-aaaf-510d4c689d8c","Type":"ContainerStarted","Data":"030cb7a19a3e8014b276c8726809943eb06ea38e110cbb408f03461d0c89c82d"}
May 28 08:13:10.523512 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:10.523475    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-network-diagnostics/network-check-target-nkgkq" event={"ID":"ecd64268-adf3-40c0-aaaf-510d4c689d8c","Type":"ContainerStarted","Data":"5ccd906f990da13b50bddc566a32d5c19925bf7afcc075c988d86b7f166f8323"}
May 28 08:13:10.523972 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:10.523690    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:13:10.539707 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:10.539642    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-network-diagnostics/network-check-target-nkgkq" podStartSLOduration=65.823231637 podStartE2EDuration="1m8.53963055s" podCreationTimestamp="2026-05-28 08:12:02 +0000 UTC" firstStartedPulling="2026-05-28 08:13:07.36076348 +0000 UTC m=+65.796942625" lastFinishedPulling="2026-05-28 08:13:10.07716239 +0000 UTC m=+68.513341538" observedRunningTime="2026-05-28 08:13:10.539266462 +0000 UTC m=+68.975445628" watchObservedRunningTime="2026-05-28 08:13:10.53963055 +0000 UTC m=+68.975809716"
May 28 08:13:12.734452 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:12.734416    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5"]
May 28 08:13:12.737277 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:12.737261    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5"
May 28 08:13:12.740204 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:12.740180    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"openshift-service-ca.crt\""
May 28 08:13:12.740312 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:12.740277    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-dockercfg-gfcbp\""
May 28 08:13:12.741675 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:12.741640    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"open-cluster-management-agent-addon\"/\"kube-root-ca.crt\""
May 28 08:13:12.741746 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:12.741685    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"open-cluster-management-image-pull-credentials\""
May 28 08:13:12.741746 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:12.741688    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"open-cluster-management-agent-addon\"/\"managed-serviceaccount-hub-kubeconfig\""
May 28 08:13:12.746007 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:12.745985    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5"]
May 28 08:13:12.898315 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:12.898281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-25s25\" (UniqueName: \"kubernetes.io/projected/7a89f0f3-541d-4200-9231-6e341f0d6951-kube-api-access-25s25\") pod \"managed-serviceaccount-addon-agent-55fc997fd4-q9dl5\" (UID: \"7a89f0f3-541d-4200-9231-6e341f0d6951\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5"
May 28 08:13:12.898509 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:12.898344    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a89f0f3-541d-4200-9231-6e341f0d6951-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55fc997fd4-q9dl5\" (UID: \"7a89f0f3-541d-4200-9231-6e341f0d6951\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5"
May 28 08:13:12.999126 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:12.999039    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-25s25\" (UniqueName: \"kubernetes.io/projected/7a89f0f3-541d-4200-9231-6e341f0d6951-kube-api-access-25s25\") pod \"managed-serviceaccount-addon-agent-55fc997fd4-q9dl5\" (UID: \"7a89f0f3-541d-4200-9231-6e341f0d6951\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5"
May 28 08:13:12.999271 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:12.999127    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a89f0f3-541d-4200-9231-6e341f0d6951-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55fc997fd4-q9dl5\" (UID: \"7a89f0f3-541d-4200-9231-6e341f0d6951\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5"
May 28 08:13:13.001588 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:13.001566    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"hub-kubeconfig\" (UniqueName: \"kubernetes.io/secret/7a89f0f3-541d-4200-9231-6e341f0d6951-hub-kubeconfig\") pod \"managed-serviceaccount-addon-agent-55fc997fd4-q9dl5\" (UID: \"7a89f0f3-541d-4200-9231-6e341f0d6951\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5"
May 28 08:13:13.012927 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:13.012907    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-25s25\" (UniqueName: \"kubernetes.io/projected/7a89f0f3-541d-4200-9231-6e341f0d6951-kube-api-access-25s25\") pod \"managed-serviceaccount-addon-agent-55fc997fd4-q9dl5\" (UID: \"7a89f0f3-541d-4200-9231-6e341f0d6951\") " pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5"
May 28 08:13:13.059528 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:13.059495    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5"
May 28 08:13:13.182503 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:13.182471    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5"]
May 28 08:13:13.185625 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:13:13.185584    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod7a89f0f3_541d_4200_9231_6e341f0d6951.slice/crio-29c7fdd49b022d222fc73d2df1c3c525efd0253d866f33c715ffaf1ee85e9a8b WatchSource:0}: Error finding container 29c7fdd49b022d222fc73d2df1c3c525efd0253d866f33c715ffaf1ee85e9a8b: Status 404 returned error can't find the container with id 29c7fdd49b022d222fc73d2df1c3c525efd0253d866f33c715ffaf1ee85e9a8b
May 28 08:13:13.530539 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:13.530504    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5" event={"ID":"7a89f0f3-541d-4200-9231-6e341f0d6951","Type":"ContainerStarted","Data":"29c7fdd49b022d222fc73d2df1c3c525efd0253d866f33c715ffaf1ee85e9a8b"}
May 28 08:13:16.538084 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:16.538045    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5" event={"ID":"7a89f0f3-541d-4200-9231-6e341f0d6951","Type":"ContainerStarted","Data":"72325b5417dac47839023efaa6cd2c18a645bc7580767c3200c5c9badb6af197"}
May 28 08:13:16.555109 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:16.555059    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="open-cluster-management-agent-addon/managed-serviceaccount-addon-agent-55fc997fd4-q9dl5" podStartSLOduration=2.128355618 podStartE2EDuration="4.555046559s" podCreationTimestamp="2026-05-28 08:13:12 +0000 UTC" firstStartedPulling="2026-05-28 08:13:13.187742838 +0000 UTC m=+71.623921989" lastFinishedPulling="2026-05-28 08:13:15.614433772 +0000 UTC m=+74.050612930" observedRunningTime="2026-05-28 08:13:16.554749781 +0000 UTC m=+74.990928946" watchObservedRunningTime="2026-05-28 08:13:16.555046559 +0000 UTC m=+74.991225725"
May 28 08:13:39.274044 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:39.273997    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:13:39.274044 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:39.274047    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:13:39.274589 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:13:39.274157    2577 secret.go:189] Couldn't get secret openshift-dns/dns-default-metrics-tls: secret "dns-default-metrics-tls" not found
May 28 08:13:39.274589 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:13:39.274156    2577 secret.go:189] Couldn't get secret openshift-ingress-canary/canary-serving-cert: secret "canary-serving-cert" not found
May 28 08:13:39.274589 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:13:39.274214    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls podName:48d7931c-9e47-424d-aaaf-1d3c90add99a nodeName:}" failed. No retries permitted until 2026-05-28 08:14:43.274199176 +0000 UTC m=+161.710378324 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "metrics-tls" (UniqueName: "kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls") pod "dns-default-vcwq4" (UID: "48d7931c-9e47-424d-aaaf-1d3c90add99a") : secret "dns-default-metrics-tls" not found
May 28 08:13:39.274589 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:13:39.274230    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert podName:50981871-e522-44dc-b0a5-209d7ec9958f nodeName:}" failed. No retries permitted until 2026-05-28 08:14:43.274222585 +0000 UTC m=+161.710401728 (durationBeforeRetry 1m4s). Error: MountVolume.SetUp failed for volume "cert" (UniqueName: "kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert") pod "ingress-canary-sjzdf" (UID: "50981871-e522-44dc-b0a5-209d7ec9958f") : secret "canary-serving-cert" not found
May 28 08:13:41.528292 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:13:41.528263    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-network-diagnostics/network-check-target-nkgkq"
May 28 08:14:10.896027 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:10.895975    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:14:10.896512 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:10.896118    2577 secret.go:189] Couldn't get secret openshift-multus/metrics-daemon-secret: secret "metrics-daemon-secret" not found
May 28 08:14:10.896512 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:10.896182    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs podName:90e9cadb-328a-47e1-a119-bae6987ac229 nodeName:}" failed. No retries permitted until 2026-05-28 08:16:12.896166429 +0000 UTC m=+251.332345572 (durationBeforeRetry 2m2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs") pod "network-metrics-daemon-mlpsc" (UID: "90e9cadb-328a-47e1-a119-bae6987ac229") : secret "metrics-daemon-secret" not found
May 28 08:14:23.226823 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.226783    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-ingress/router-default-7df5fc48dd-7nhw9"]
May 28 08:14:23.229828 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.229808    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-xkwsz"]
May 28 08:14:23.229978 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.229960    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.232496 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.232470    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.232496 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.232485    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-dockercfg-g9hp8\""
May 28 08:14:23.233789 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.233771    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"service-ca-bundle\""
May 28 08:14:23.236342 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.236326    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"default-ingress-cert\""
May 28 08:14:23.237202 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.237051    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-metrics-certs-default\""
May 28 08:14:23.237202 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.237117    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"kube-root-ca.crt\""
May 28 08:14:23.237317 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.237302    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-ingress\"/\"openshift-service-ca.crt\""
May 28 08:14:23.237423 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.237410    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress\"/\"router-stats-default\""
May 28 08:14:23.237632 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.237613    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"kube-root-ca.crt\""
May 28 08:14:23.237749 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.237670    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"openshift-service-ca.crt\""
May 28 08:14:23.237749 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.237711    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"console-operator-config\""
May 28 08:14:23.237749 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.237613    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"serving-cert\""
May 28 08:14:23.237943 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.237799    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console-operator\"/\"console-operator-dockercfg-pq7s7\""
May 28 08:14:23.242923 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.242902    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console-operator\"/\"trusted-ca\""
May 28 08:14:23.256473 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.256447    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-xkwsz"]
May 28 08:14:23.260851 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.260827    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7df5fc48dd-7nhw9"]
May 28 08:14:23.277996 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.277966    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/601ae44f-2916-47d8-9538-61dd910c49d7-serving-cert\") pod \"console-operator-76dff48d59-xkwsz\" (UID: \"601ae44f-2916-47d8-9538-61dd910c49d7\") " pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.277996 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.277999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/601ae44f-2916-47d8-9538-61dd910c49d7-trusted-ca\") pod \"console-operator-76dff48d59-xkwsz\" (UID: \"601ae44f-2916-47d8-9538-61dd910c49d7\") " pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.278196 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.278026    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.278196 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.278090    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.278196 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.278108    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-stats-auth\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.278196 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.278132    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/601ae44f-2916-47d8-9538-61dd910c49d7-config\") pod \"console-operator-76dff48d59-xkwsz\" (UID: \"601ae44f-2916-47d8-9538-61dd910c49d7\") " pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.278196 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.278147    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-5dw6l\" (UniqueName: \"kubernetes.io/projected/601ae44f-2916-47d8-9538-61dd910c49d7-kube-api-access-5dw6l\") pod \"console-operator-76dff48d59-xkwsz\" (UID: \"601ae44f-2916-47d8-9538-61dd910c49d7\") " pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.278196 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.278167    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-default-certificate\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.278399 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.278237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dnd9h\" (UniqueName: \"kubernetes.io/projected/2b55796c-941d-42d5-a985-40d51c23e675-kube-api-access-dnd9h\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.325799 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.325766    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"]
May 28 08:14:23.328540 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.328519    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"
May 28 08:14:23.329077 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.329052    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-image-registry/image-registry-798b45b98d-ctqd2"]
May 28 08:14:23.331189 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.331174    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-storage-version-migrator-operator-dockercfg-qvbpj\""
May 28 08:14:23.331538 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.331521    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.331616 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.331567    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"config\""
May 28 08:14:23.331616 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.331589    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"openshift-service-ca.crt\""
May 28 08:14:23.332300 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.332284    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"serving-cert\""
May 28 08:14:23.332396 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.332383    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-kube-storage-version-migrator-operator\"/\"kube-root-ca.crt\""
May 28 08:14:23.334303 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.334288    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-private-configuration\""
May 28 08:14:23.334577 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.334560    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"installation-pull-secrets\""
May 28 08:14:23.334647 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.334618    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"registry-dockercfg-8b8v4\""
May 28 08:14:23.335188 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.335171    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-image-registry\"/\"image-registry-tls\""
May 28 08:14:23.341808 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.341786    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-image-registry\"/\"trusted-ca\""
May 28 08:14:23.347156 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.347129    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"]
May 28 08:14:23.347955 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.347935    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-798b45b98d-ctqd2"]
May 28 08:14:23.378914 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.378882    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dnd9h\" (UniqueName: \"kubernetes.io/projected/2b55796c-941d-42d5-a985-40d51c23e675-kube-api-access-dnd9h\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.378914 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.378922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-ca-trust-extracted\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.379136 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.378957    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/601ae44f-2916-47d8-9538-61dd910c49d7-serving-cert\") pod \"console-operator-76dff48d59-xkwsz\" (UID: \"601ae44f-2916-47d8-9538-61dd910c49d7\") " pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.379136 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.378985    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-bound-sa-token\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.379136 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379022    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/601ae44f-2916-47d8-9538-61dd910c49d7-trusted-ca\") pod \"console-operator-76dff48d59-xkwsz\" (UID: \"601ae44f-2916-47d8-9538-61dd910c49d7\") " pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.379136 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379043    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-image-registry-private-configuration\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.379136 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379069    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.379136 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379096    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-trusted-ca\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.379136 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379133    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dfrz6\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-kube-api-access-dfrz6\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.379455 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379171    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.379455 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:23.379185    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 28 08:14:23.379455 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379200    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-stats-auth\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.379455 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:23.379303    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs podName:2b55796c-941d-42d5-a985-40d51c23e675 nodeName:}" failed. No retries permitted until 2026-05-28 08:14:23.879269428 +0000 UTC m=+142.315448586 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs") pod "router-default-7df5fc48dd-7nhw9" (UID: "2b55796c-941d-42d5-a985-40d51c23e675") : secret "router-metrics-certs-default" not found
May 28 08:14:23.379455 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:23.379380    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle podName:2b55796c-941d-42d5-a985-40d51c23e675 nodeName:}" failed. No retries permitted until 2026-05-28 08:14:23.879360516 +0000 UTC m=+142.315539690 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle") pod "router-default-7df5fc48dd-7nhw9" (UID: "2b55796c-941d-42d5-a985-40d51c23e675") : configmap references non-existent config key: service-ca.crt
May 28 08:14:23.379455 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379414    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/601ae44f-2916-47d8-9538-61dd910c49d7-config\") pod \"console-operator-76dff48d59-xkwsz\" (UID: \"601ae44f-2916-47d8-9538-61dd910c49d7\") " pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.379455 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379447    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-5dw6l\" (UniqueName: \"kubernetes.io/projected/601ae44f-2916-47d8-9538-61dd910c49d7-kube-api-access-5dw6l\") pod \"console-operator-76dff48d59-xkwsz\" (UID: \"601ae44f-2916-47d8-9538-61dd910c49d7\") " pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.379839 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379470    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.379839 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379495    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/596d1656-db0e-41c6-8dc1-9926d1b470c9-config\") pod \"kube-storage-version-migrator-operator-565644f49-qzjtr\" (UID: \"596d1656-db0e-41c6-8dc1-9926d1b470c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"
May 28 08:14:23.379839 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379520    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-certificates\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.379839 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379554    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-default-certificate\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.379839 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379581    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-installation-pull-secrets\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.379839 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379685    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/596d1656-db0e-41c6-8dc1-9926d1b470c9-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-qzjtr\" (UID: \"596d1656-db0e-41c6-8dc1-9926d1b470c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"
May 28 08:14:23.379839 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.379719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-b4tz4\" (UniqueName: \"kubernetes.io/projected/596d1656-db0e-41c6-8dc1-9926d1b470c9-kube-api-access-b4tz4\") pod \"kube-storage-version-migrator-operator-565644f49-qzjtr\" (UID: \"596d1656-db0e-41c6-8dc1-9926d1b470c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"
May 28 08:14:23.380349 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.380113    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/601ae44f-2916-47d8-9538-61dd910c49d7-config\") pod \"console-operator-76dff48d59-xkwsz\" (UID: \"601ae44f-2916-47d8-9538-61dd910c49d7\") " pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.380409 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.380362    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/601ae44f-2916-47d8-9538-61dd910c49d7-trusted-ca\") pod \"console-operator-76dff48d59-xkwsz\" (UID: \"601ae44f-2916-47d8-9538-61dd910c49d7\") " pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.381581 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.381556    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/601ae44f-2916-47d8-9538-61dd910c49d7-serving-cert\") pod \"console-operator-76dff48d59-xkwsz\" (UID: \"601ae44f-2916-47d8-9538-61dd910c49d7\") " pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.381833 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.381811    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"stats-auth\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-stats-auth\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.382100 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.382080    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"default-certificate\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-default-certificate\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.390833 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.390800    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-5dw6l\" (UniqueName: \"kubernetes.io/projected/601ae44f-2916-47d8-9538-61dd910c49d7-kube-api-access-5dw6l\") pod \"console-operator-76dff48d59-xkwsz\" (UID: \"601ae44f-2916-47d8-9538-61dd910c49d7\") " pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.390981 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.390961    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dnd9h\" (UniqueName: \"kubernetes.io/projected/2b55796c-941d-42d5-a985-40d51c23e675-kube-api-access-dnd9h\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.480245 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480164    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.480245 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480200    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/596d1656-db0e-41c6-8dc1-9926d1b470c9-config\") pod \"kube-storage-version-migrator-operator-565644f49-qzjtr\" (UID: \"596d1656-db0e-41c6-8dc1-9926d1b470c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"
May 28 08:14:23.480245 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480216    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-certificates\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.480245 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480235    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-installation-pull-secrets\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.480543 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480268    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/596d1656-db0e-41c6-8dc1-9926d1b470c9-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-qzjtr\" (UID: \"596d1656-db0e-41c6-8dc1-9926d1b470c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"
May 28 08:14:23.480543 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480285    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-b4tz4\" (UniqueName: \"kubernetes.io/projected/596d1656-db0e-41c6-8dc1-9926d1b470c9-kube-api-access-b4tz4\") pod \"kube-storage-version-migrator-operator-565644f49-qzjtr\" (UID: \"596d1656-db0e-41c6-8dc1-9926d1b470c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"
May 28 08:14:23.480543 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480302    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-ca-trust-extracted\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.480543 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:23.480308    2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 08:14:23.480543 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:23.480329    2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798b45b98d-ctqd2: secret "image-registry-tls" not found
May 28 08:14:23.480543 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480330    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-bound-sa-token\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.480543 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480360    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-image-registry-private-configuration\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.480543 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:23.480405    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls podName:fa061e87-5cd2-4e3c-b6b0-eed405427b1c nodeName:}" failed. No retries permitted until 2026-05-28 08:14:23.980385078 +0000 UTC m=+142.416564244 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls") pod "image-registry-798b45b98d-ctqd2" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c") : secret "image-registry-tls" not found
May 28 08:14:23.480543 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480446    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-trusted-ca\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.480543 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480491    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dfrz6\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-kube-api-access-dfrz6\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.481048 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480843    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/596d1656-db0e-41c6-8dc1-9926d1b470c9-config\") pod \"kube-storage-version-migrator-operator-565644f49-qzjtr\" (UID: \"596d1656-db0e-41c6-8dc1-9926d1b470c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"
May 28 08:14:23.481048 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.480917    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-certificates\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.481128 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.481080    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-ca-trust-extracted\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.481498 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.481476    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-trusted-ca\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.482824 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.482804    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/596d1656-db0e-41c6-8dc1-9926d1b470c9-serving-cert\") pod \"kube-storage-version-migrator-operator-565644f49-qzjtr\" (UID: \"596d1656-db0e-41c6-8dc1-9926d1b470c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"
May 28 08:14:23.483060 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.483040    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-image-registry-private-configuration\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.483237 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.483220    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-installation-pull-secrets\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.489846 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.489820    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-bound-sa-token\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.490426 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.490401    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-b4tz4\" (UniqueName: \"kubernetes.io/projected/596d1656-db0e-41c6-8dc1-9926d1b470c9-kube-api-access-b4tz4\") pod \"kube-storage-version-migrator-operator-565644f49-qzjtr\" (UID: \"596d1656-db0e-41c6-8dc1-9926d1b470c9\") " pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"
May 28 08:14:23.490588 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.490571    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dfrz6\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-kube-api-access-dfrz6\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.545800 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.545758    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:23.639997 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.639966    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"
May 28 08:14:23.667898 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.667868    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console-operator/console-operator-76dff48d59-xkwsz"]
May 28 08:14:23.671451 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:14:23.671421    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod601ae44f_2916_47d8_9538_61dd910c49d7.slice/crio-6ef81536a225cde3415636cbf12ba98f76fe7729ace7576476495ec9310111cc WatchSource:0}: Error finding container 6ef81536a225cde3415636cbf12ba98f76fe7729ace7576476495ec9310111cc: Status 404 returned error can't find the container with id 6ef81536a225cde3415636cbf12ba98f76fe7729ace7576476495ec9310111cc
May 28 08:14:23.771352 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.771328    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr"]
May 28 08:14:23.773507 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:14:23.773474    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod596d1656_db0e_41c6_8dc1_9926d1b470c9.slice/crio-860752ef27608160901bc8e5b5189295aba364be54ff80b7b148137eb51eed83 WatchSource:0}: Error finding container 860752ef27608160901bc8e5b5189295aba364be54ff80b7b148137eb51eed83: Status 404 returned error can't find the container with id 860752ef27608160901bc8e5b5189295aba364be54ff80b7b148137eb51eed83
May 28 08:14:23.883628 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.883590    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.883840 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.883638    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:23.883840 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:23.883785    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 28 08:14:23.883840 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:23.883824    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle podName:2b55796c-941d-42d5-a985-40d51c23e675 nodeName:}" failed. No retries permitted until 2026-05-28 08:14:24.883807366 +0000 UTC m=+143.319986510 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle") pod "router-default-7df5fc48dd-7nhw9" (UID: "2b55796c-941d-42d5-a985-40d51c23e675") : configmap references non-existent config key: service-ca.crt
May 28 08:14:23.883968 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:23.883853    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs podName:2b55796c-941d-42d5-a985-40d51c23e675 nodeName:}" failed. No retries permitted until 2026-05-28 08:14:24.883835528 +0000 UTC m=+143.320014688 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs") pod "router-default-7df5fc48dd-7nhw9" (UID: "2b55796c-941d-42d5-a985-40d51c23e675") : secret "router-metrics-certs-default" not found
May 28 08:14:23.984714 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:23.984671    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:23.984887 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:23.984813    2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 08:14:23.984887 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:23.984836    2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798b45b98d-ctqd2: secret "image-registry-tls" not found
May 28 08:14:23.984971 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:23.984895    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls podName:fa061e87-5cd2-4e3c-b6b0-eed405427b1c nodeName:}" failed. No retries permitted until 2026-05-28 08:14:24.984876653 +0000 UTC m=+143.421055803 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls") pod "image-registry-798b45b98d-ctqd2" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c") : secret "image-registry-tls" not found
May 28 08:14:24.668390 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:24.668346    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr" event={"ID":"596d1656-db0e-41c6-8dc1-9926d1b470c9","Type":"ContainerStarted","Data":"860752ef27608160901bc8e5b5189295aba364be54ff80b7b148137eb51eed83"}
May 28 08:14:24.669605 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:24.669532    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" event={"ID":"601ae44f-2916-47d8-9538-61dd910c49d7","Type":"ContainerStarted","Data":"6ef81536a225cde3415636cbf12ba98f76fe7729ace7576476495ec9310111cc"}
May 28 08:14:24.892512 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:24.892472    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:24.892702 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:24.892600    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:24.892702 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:24.892646    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle podName:2b55796c-941d-42d5-a985-40d51c23e675 nodeName:}" failed. No retries permitted until 2026-05-28 08:14:26.892625646 +0000 UTC m=+145.328804843 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle") pod "router-default-7df5fc48dd-7nhw9" (UID: "2b55796c-941d-42d5-a985-40d51c23e675") : configmap references non-existent config key: service-ca.crt
May 28 08:14:24.892792 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:24.892719    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 28 08:14:24.892792 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:24.892760    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs podName:2b55796c-941d-42d5-a985-40d51c23e675 nodeName:}" failed. No retries permitted until 2026-05-28 08:14:26.892746403 +0000 UTC m=+145.328925549 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs") pod "router-default-7df5fc48dd-7nhw9" (UID: "2b55796c-941d-42d5-a985-40d51c23e675") : secret "router-metrics-certs-default" not found
May 28 08:14:24.993924 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:24.993711    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:24.993924 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:24.993873    2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 08:14:24.993924 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:24.993893    2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798b45b98d-ctqd2: secret "image-registry-tls" not found
May 28 08:14:24.994255 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:24.993953    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls podName:fa061e87-5cd2-4e3c-b6b0-eed405427b1c nodeName:}" failed. No retries permitted until 2026-05-28 08:14:26.993936555 +0000 UTC m=+145.430115702 (durationBeforeRetry 2s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls") pod "image-registry-798b45b98d-ctqd2" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c") : secret "image-registry-tls" not found
May 28 08:14:25.031944 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.031904    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"]
May 28 08:14:25.034616 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.034592    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"
May 28 08:14:25.037453 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.037428    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"serving-cert\""
May 28 08:14:25.037577 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.037466    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"kube-root-ca.crt\""
May 28 08:14:25.037577 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.037503    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-config\""
May 28 08:14:25.037577 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.037511    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca-operator\"/\"openshift-service-ca.crt\""
May 28 08:14:25.038501 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.038487    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca-operator\"/\"service-ca-operator-dockercfg-rscfj\""
May 28 08:14:25.044666 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.044620    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"]
May 28 08:14:25.094413 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.094366    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff8e5102-0c6f-4fd5-a002-7ad71a644f3b-config\") pod \"service-ca-operator-57b5b68b8-zgn4r\" (UID: \"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"
May 28 08:14:25.094413 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.094418    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff8e5102-0c6f-4fd5-a002-7ad71a644f3b-serving-cert\") pod \"service-ca-operator-57b5b68b8-zgn4r\" (UID: \"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"
May 28 08:14:25.094678 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.094449    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9gjh\" (UniqueName: \"kubernetes.io/projected/ff8e5102-0c6f-4fd5-a002-7ad71a644f3b-kube-api-access-p9gjh\") pod \"service-ca-operator-57b5b68b8-zgn4r\" (UID: \"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"
May 28 08:14:25.195311 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.195264    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff8e5102-0c6f-4fd5-a002-7ad71a644f3b-config\") pod \"service-ca-operator-57b5b68b8-zgn4r\" (UID: \"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"
May 28 08:14:25.195512 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.195321    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff8e5102-0c6f-4fd5-a002-7ad71a644f3b-serving-cert\") pod \"service-ca-operator-57b5b68b8-zgn4r\" (UID: \"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"
May 28 08:14:25.195512 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.195356    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p9gjh\" (UniqueName: \"kubernetes.io/projected/ff8e5102-0c6f-4fd5-a002-7ad71a644f3b-kube-api-access-p9gjh\") pod \"service-ca-operator-57b5b68b8-zgn4r\" (UID: \"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"
May 28 08:14:25.195957 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.195924    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config\" (UniqueName: \"kubernetes.io/configmap/ff8e5102-0c6f-4fd5-a002-7ad71a644f3b-config\") pod \"service-ca-operator-57b5b68b8-zgn4r\" (UID: \"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"
May 28 08:14:25.198560 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.198534    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"serving-cert\" (UniqueName: \"kubernetes.io/secret/ff8e5102-0c6f-4fd5-a002-7ad71a644f3b-serving-cert\") pod \"service-ca-operator-57b5b68b8-zgn4r\" (UID: \"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"
May 28 08:14:25.204954 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.204929    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9gjh\" (UniqueName: \"kubernetes.io/projected/ff8e5102-0c6f-4fd5-a002-7ad71a644f3b-kube-api-access-p9gjh\") pod \"service-ca-operator-57b5b68b8-zgn4r\" (UID: \"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b\") " pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"
May 28 08:14:25.343883 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.343841    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"
May 28 08:14:25.673283 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.673241    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" event={"ID":"601ae44f-2916-47d8-9538-61dd910c49d7","Type":"ContainerStarted","Data":"21f35655d4b7093d3acd247ebc91e1de6fed79e8799a4820c5387b8a1f7e3726"}
May 28 08:14:25.673747 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.673387    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:25.675093 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.675064    2577 patch_prober.go:28] interesting pod/console-operator-76dff48d59-xkwsz container/console-operator namespace/openshift-console-operator: Readiness probe status=failure output="Get \"https://10.134.0.9:8443/readyz\": dial tcp 10.134.0.9:8443: connect: connection refused" start-of-body=
May 28 08:14:25.675180 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.675149    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" podUID="601ae44f-2916-47d8-9538-61dd910c49d7" containerName="console-operator" probeResult="failure" output="Get \"https://10.134.0.9:8443/readyz\": dial tcp 10.134.0.9:8443: connect: connection refused"
May 28 08:14:25.681310 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.681253    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r"]
May 28 08:14:25.692579 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:25.692366    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" podStartSLOduration=0.765005433 podStartE2EDuration="2.692347135s" podCreationTimestamp="2026-05-28 08:14:23 +0000 UTC" firstStartedPulling="2026-05-28 08:14:23.673171502 +0000 UTC m=+142.109350647" lastFinishedPulling="2026-05-28 08:14:25.600513195 +0000 UTC m=+144.036692349" observedRunningTime="2026-05-28 08:14:25.689390969 +0000 UTC m=+144.125570135" watchObservedRunningTime="2026-05-28 08:14:25.692347135 +0000 UTC m=+144.128526301"
May 28 08:14:26.097550 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:14:26.097509    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podff8e5102_0c6f_4fd5_a002_7ad71a644f3b.slice/crio-e672733800068332ac4979f683d976ccfce0fcd24998f83f9256009df9c29787 WatchSource:0}: Error finding container e672733800068332ac4979f683d976ccfce0fcd24998f83f9256009df9c29787: Status 404 returned error can't find the container with id e672733800068332ac4979f683d976ccfce0fcd24998f83f9256009df9c29787
May 28 08:14:26.676731 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:26.676691    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r" event={"ID":"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b","Type":"ContainerStarted","Data":"e672733800068332ac4979f683d976ccfce0fcd24998f83f9256009df9c29787"}
May 28 08:14:26.678467 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:26.678438    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr" event={"ID":"596d1656-db0e-41c6-8dc1-9926d1b470c9","Type":"ContainerStarted","Data":"5a0bba6828ba41106ca10f26a1d0fafdcbf34c9e48d215714a7e9201f7e1a9f1"}
May 28 08:14:26.679905 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:26.679778    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/0.log"
May 28 08:14:26.679905 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:26.679810    2577 generic.go:358] "Generic (PLEG): container finished" podID="601ae44f-2916-47d8-9538-61dd910c49d7" containerID="21f35655d4b7093d3acd247ebc91e1de6fed79e8799a4820c5387b8a1f7e3726" exitCode=255
May 28 08:14:26.679905 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:26.679867    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" event={"ID":"601ae44f-2916-47d8-9538-61dd910c49d7","Type":"ContainerDied","Data":"21f35655d4b7093d3acd247ebc91e1de6fed79e8799a4820c5387b8a1f7e3726"}
May 28 08:14:26.680209 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:26.680194    2577 scope.go:117] "RemoveContainer" containerID="21f35655d4b7093d3acd247ebc91e1de6fed79e8799a4820c5387b8a1f7e3726"
May 28 08:14:26.694966 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:26.694909    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr" podStartSLOduration=1.325127302 podStartE2EDuration="3.694890172s" podCreationTimestamp="2026-05-28 08:14:23 +0000 UTC" firstStartedPulling="2026-05-28 08:14:23.775264116 +0000 UTC m=+142.211443260" lastFinishedPulling="2026-05-28 08:14:26.145026984 +0000 UTC m=+144.581206130" observedRunningTime="2026-05-28 08:14:26.693995554 +0000 UTC m=+145.130174721" watchObservedRunningTime="2026-05-28 08:14:26.694890172 +0000 UTC m=+145.131069339"
May 28 08:14:26.909119 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:26.909083    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:26.909280 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:26.909170    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:26.909352 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:26.909273    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle podName:2b55796c-941d-42d5-a985-40d51c23e675 nodeName:}" failed. No retries permitted until 2026-05-28 08:14:30.90924674 +0000 UTC m=+149.345425945 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle") pod "router-default-7df5fc48dd-7nhw9" (UID: "2b55796c-941d-42d5-a985-40d51c23e675") : configmap references non-existent config key: service-ca.crt
May 28 08:14:26.909401 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:26.909346    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 28 08:14:26.909435 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:26.909416    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs podName:2b55796c-941d-42d5-a985-40d51c23e675 nodeName:}" failed. No retries permitted until 2026-05-28 08:14:30.909398318 +0000 UTC m=+149.345577467 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs") pod "router-default-7df5fc48dd-7nhw9" (UID: "2b55796c-941d-42d5-a985-40d51c23e675") : secret "router-metrics-certs-default" not found
May 28 08:14:27.010024 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:27.009987    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:27.010216 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:27.010154    2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 08:14:27.010216 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:27.010176    2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798b45b98d-ctqd2: secret "image-registry-tls" not found
May 28 08:14:27.010333 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:27.010244    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls podName:fa061e87-5cd2-4e3c-b6b0-eed405427b1c nodeName:}" failed. No retries permitted until 2026-05-28 08:14:31.010224186 +0000 UTC m=+149.446403349 (durationBeforeRetry 4s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls") pod "image-registry-798b45b98d-ctqd2" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c") : secret "image-registry-tls" not found
May 28 08:14:27.682882 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:27.682854    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/1.log"
May 28 08:14:27.683332 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:27.683299    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/0.log"
May 28 08:14:27.683384 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:27.683342    2577 generic.go:358] "Generic (PLEG): container finished" podID="601ae44f-2916-47d8-9538-61dd910c49d7" containerID="32c676400f3e12dc22cc1226c402f4bd42bc6aa0101c5b4e03ad39480a904281" exitCode=255
May 28 08:14:27.683474 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:27.683441    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" event={"ID":"601ae44f-2916-47d8-9538-61dd910c49d7","Type":"ContainerDied","Data":"32c676400f3e12dc22cc1226c402f4bd42bc6aa0101c5b4e03ad39480a904281"}
May 28 08:14:27.683524 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:27.683499    2577 scope.go:117] "RemoveContainer" containerID="21f35655d4b7093d3acd247ebc91e1de6fed79e8799a4820c5387b8a1f7e3726"
May 28 08:14:27.684181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:27.683694    2577 scope.go:117] "RemoveContainer" containerID="32c676400f3e12dc22cc1226c402f4bd42bc6aa0101c5b4e03ad39480a904281"
May 28 08:14:27.684181 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:27.683952    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-xkwsz_openshift-console-operator(601ae44f-2916-47d8-9538-61dd910c49d7)\"" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" podUID="601ae44f-2916-47d8-9538-61dd910c49d7"
May 28 08:14:28.686246 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:28.686210    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r" event={"ID":"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b","Type":"ContainerStarted","Data":"2365dbfc95b2c3cb26d6772e5205e3118fec0eca6af75035d53aca164d26cb50"}
May 28 08:14:28.687484 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:28.687465    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/1.log"
May 28 08:14:28.687773 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:28.687759    2577 scope.go:117] "RemoveContainer" containerID="32c676400f3e12dc22cc1226c402f4bd42bc6aa0101c5b4e03ad39480a904281"
May 28 08:14:28.687914 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:28.687898    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-xkwsz_openshift-console-operator(601ae44f-2916-47d8-9538-61dd910c49d7)\"" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" podUID="601ae44f-2916-47d8-9538-61dd910c49d7"
May 28 08:14:28.701899 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:28.701852    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r" podStartSLOduration=1.7063717299999999 podStartE2EDuration="3.701839403s" podCreationTimestamp="2026-05-28 08:14:25 +0000 UTC" firstStartedPulling="2026-05-28 08:14:26.099613126 +0000 UTC m=+144.535792269" lastFinishedPulling="2026-05-28 08:14:28.095080796 +0000 UTC m=+146.531259942" observedRunningTime="2026-05-28 08:14:28.700801418 +0000 UTC m=+147.136980583" watchObservedRunningTime="2026-05-28 08:14:28.701839403 +0000 UTC m=+147.138018563"
May 28 08:14:30.123004 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:30.122972    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k9tpr_07b247ee-4fc0-4b9e-ae81-9f260ce894fc/dns-node-resolver/0.log"
May 28 08:14:30.943797 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:30.943763    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:30.943977 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:30.943836    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:30.943977 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:30.943942    2577 secret.go:189] Couldn't get secret openshift-ingress/router-metrics-certs-default: secret "router-metrics-certs-default" not found
May 28 08:14:30.943977 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:30.943970    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle podName:2b55796c-941d-42d5-a985-40d51c23e675 nodeName:}" failed. No retries permitted until 2026-05-28 08:14:38.943955621 +0000 UTC m=+157.380134766 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle") pod "router-default-7df5fc48dd-7nhw9" (UID: "2b55796c-941d-42d5-a985-40d51c23e675") : configmap references non-existent config key: service-ca.crt
May 28 08:14:30.944098 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:30.944010    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs podName:2b55796c-941d-42d5-a985-40d51c23e675 nodeName:}" failed. No retries permitted until 2026-05-28 08:14:38.94399089 +0000 UTC m=+157.380170037 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "metrics-certs" (UniqueName: "kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs") pod "router-default-7df5fc48dd-7nhw9" (UID: "2b55796c-941d-42d5-a985-40d51c23e675") : secret "router-metrics-certs-default" not found
May 28 08:14:31.044635 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.044600    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:31.044818 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:31.044763    2577 projected.go:264] Couldn't get secret openshift-image-registry/image-registry-tls: secret "image-registry-tls" not found
May 28 08:14:31.044818 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:31.044784    2577 projected.go:194] Error preparing data for projected volume registry-tls for pod openshift-image-registry/image-registry-798b45b98d-ctqd2: secret "image-registry-tls" not found
May 28 08:14:31.044906 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:31.044841    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls podName:fa061e87-5cd2-4e3c-b6b0-eed405427b1c nodeName:}" failed. No retries permitted until 2026-05-28 08:14:39.044822988 +0000 UTC m=+157.481002136 (durationBeforeRetry 8s). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls") pod "image-registry-798b45b98d-ctqd2" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c") : secret "image-registry-tls" not found
May 28 08:14:31.321028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.320999    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bp4g8_a4b772f7-1b71-4e3b-b2a9-fbf33edfce66/node-ca/0.log"
May 28 08:14:31.777878 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.777840    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-q27h4"]
May 28 08:14:31.780786 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.780769    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-q27h4"
May 28 08:14:31.786450 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.786424    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"openshift-service-ca.crt\""
May 28 08:14:31.786450 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.786447    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"signing-cabundle\""
May 28 08:14:31.786685 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.786472    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-service-ca\"/\"kube-root-ca.crt\""
May 28 08:14:31.787571 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.787553    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"service-ca-dockercfg-fswwl\""
May 28 08:14:31.788552 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.788534    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-service-ca\"/\"signing-key\""
May 28 08:14:31.792789 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.792766    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-q27h4"]
May 28 08:14:31.849781 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.849746    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fdb34d0a-27e5-459c-aaca-85334149a96f-signing-cabundle\") pod \"service-ca-67d667c7d-q27h4\" (UID: \"fdb34d0a-27e5-459c-aaca-85334149a96f\") " pod="openshift-service-ca/service-ca-67d667c7d-q27h4"
May 28 08:14:31.849980 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.849835    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fdb34d0a-27e5-459c-aaca-85334149a96f-signing-key\") pod \"service-ca-67d667c7d-q27h4\" (UID: \"fdb34d0a-27e5-459c-aaca-85334149a96f\") " pod="openshift-service-ca/service-ca-67d667c7d-q27h4"
May 28 08:14:31.849980 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.849957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-bppwq\" (UniqueName: \"kubernetes.io/projected/fdb34d0a-27e5-459c-aaca-85334149a96f-kube-api-access-bppwq\") pod \"service-ca-67d667c7d-q27h4\" (UID: \"fdb34d0a-27e5-459c-aaca-85334149a96f\") " pod="openshift-service-ca/service-ca-67d667c7d-q27h4"
May 28 08:14:31.951126 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.951088    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-bppwq\" (UniqueName: \"kubernetes.io/projected/fdb34d0a-27e5-459c-aaca-85334149a96f-kube-api-access-bppwq\") pod \"service-ca-67d667c7d-q27h4\" (UID: \"fdb34d0a-27e5-459c-aaca-85334149a96f\") " pod="openshift-service-ca/service-ca-67d667c7d-q27h4"
May 28 08:14:31.951317 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.951132    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fdb34d0a-27e5-459c-aaca-85334149a96f-signing-cabundle\") pod \"service-ca-67d667c7d-q27h4\" (UID: \"fdb34d0a-27e5-459c-aaca-85334149a96f\") " pod="openshift-service-ca/service-ca-67d667c7d-q27h4"
May 28 08:14:31.951317 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.951233    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fdb34d0a-27e5-459c-aaca-85334149a96f-signing-key\") pod \"service-ca-67d667c7d-q27h4\" (UID: \"fdb34d0a-27e5-459c-aaca-85334149a96f\") " pod="openshift-service-ca/service-ca-67d667c7d-q27h4"
May 28 08:14:31.951809 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.951780    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-cabundle\" (UniqueName: \"kubernetes.io/configmap/fdb34d0a-27e5-459c-aaca-85334149a96f-signing-cabundle\") pod \"service-ca-67d667c7d-q27h4\" (UID: \"fdb34d0a-27e5-459c-aaca-85334149a96f\") " pod="openshift-service-ca/service-ca-67d667c7d-q27h4"
May 28 08:14:31.953768 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.953744    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"signing-key\" (UniqueName: \"kubernetes.io/secret/fdb34d0a-27e5-459c-aaca-85334149a96f-signing-key\") pod \"service-ca-67d667c7d-q27h4\" (UID: \"fdb34d0a-27e5-459c-aaca-85334149a96f\") " pod="openshift-service-ca/service-ca-67d667c7d-q27h4"
May 28 08:14:31.960586 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:31.960563    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-bppwq\" (UniqueName: \"kubernetes.io/projected/fdb34d0a-27e5-459c-aaca-85334149a96f-kube-api-access-bppwq\") pod \"service-ca-67d667c7d-q27h4\" (UID: \"fdb34d0a-27e5-459c-aaca-85334149a96f\") " pod="openshift-service-ca/service-ca-67d667c7d-q27h4"
May 28 08:14:32.089601 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:32.089523    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-service-ca/service-ca-67d667c7d-q27h4"
May 28 08:14:32.222745 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:32.222712    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-service-ca/service-ca-67d667c7d-q27h4"]
May 28 08:14:32.225748 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:14:32.225719    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfdb34d0a_27e5_459c_aaca_85334149a96f.slice/crio-078d03bb3f200ce1e0956c57bc2a2b6c8ccada9715f3fe319b57b9065f0e843e WatchSource:0}: Error finding container 078d03bb3f200ce1e0956c57bc2a2b6c8ccada9715f3fe319b57b9065f0e843e: Status 404 returned error can't find the container with id 078d03bb3f200ce1e0956c57bc2a2b6c8ccada9715f3fe319b57b9065f0e843e
May 28 08:14:32.696313 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:32.696272    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-q27h4" event={"ID":"fdb34d0a-27e5-459c-aaca-85334149a96f","Type":"ContainerStarted","Data":"6dc534a5f06e5788acf7d928221cc3eed2f696a2a3f92417a3735ac9731b3454"}
May 28 08:14:32.696313 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:32.696311    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca/service-ca-67d667c7d-q27h4" event={"ID":"fdb34d0a-27e5-459c-aaca-85334149a96f","Type":"ContainerStarted","Data":"078d03bb3f200ce1e0956c57bc2a2b6c8ccada9715f3fe319b57b9065f0e843e"}
May 28 08:14:32.712830 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:32.712774    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-service-ca/service-ca-67d667c7d-q27h4" podStartSLOduration=1.7127567620000002 podStartE2EDuration="1.712756762s" podCreationTimestamp="2026-05-28 08:14:31 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:14:32.7125935 +0000 UTC m=+151.148772667" watchObservedRunningTime="2026-05-28 08:14:32.712756762 +0000 UTC m=+151.148935931"
May 28 08:14:32.921846 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:32.921814    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-qzjtr_596d1656-db0e-41c6-8dc1-9926d1b470c9/kube-storage-version-migrator-operator/0.log"
May 28 08:14:33.546256 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:33.546210    2577 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:33.546747 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:33.546731    2577 scope.go:117] "RemoveContainer" containerID="32c676400f3e12dc22cc1226c402f4bd42bc6aa0101c5b4e03ad39480a904281"
May 28 08:14:33.546993 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:33.546940    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-xkwsz_openshift-console-operator(601ae44f-2916-47d8-9538-61dd910c49d7)\"" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" podUID="601ae44f-2916-47d8-9538-61dd910c49d7"
May 28 08:14:35.673933 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:35.673888    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:35.674373 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:35.674358    2577 scope.go:117] "RemoveContainer" containerID="32c676400f3e12dc22cc1226c402f4bd42bc6aa0101c5b4e03ad39480a904281"
May 28 08:14:35.674586 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:35.674566    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 10s restarting failed container=console-operator pod=console-operator-76dff48d59-xkwsz_openshift-console-operator(601ae44f-2916-47d8-9538-61dd910c49d7)\"" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" podUID="601ae44f-2916-47d8-9538-61dd910c49d7"
May 28 08:14:38.439669 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:38.439595    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[cert], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-ingress-canary/ingress-canary-sjzdf" podUID="50981871-e522-44dc-b0a5-209d7ec9958f"
May 28 08:14:38.445810 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:38.445775    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-tls], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-dns/dns-default-vcwq4" podUID="48d7931c-9e47-424d-aaaf-1d3c90add99a"
May 28 08:14:38.711035 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:38.710960    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vcwq4"
May 28 08:14:39.005025 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:39.004938    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:39.005025 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:39.005026    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:39.005213 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:39.005174    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle podName:2b55796c-941d-42d5-a985-40d51c23e675 nodeName:}" failed. No retries permitted until 2026-05-28 08:14:55.005160002 +0000 UTC m=+173.441339145 (durationBeforeRetry 16s). Error: MountVolume.SetUp failed for volume "service-ca-bundle" (UniqueName: "kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle") pod "router-default-7df5fc48dd-7nhw9" (UID: "2b55796c-941d-42d5-a985-40d51c23e675") : configmap references non-existent config key: service-ca.crt
May 28 08:14:39.007539 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:39.007505    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/2b55796c-941d-42d5-a985-40d51c23e675-metrics-certs\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:39.106535 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:39.106491    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:39.109138 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:39.109107    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls\") pod \"image-registry-798b45b98d-ctqd2\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") " pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:39.245630 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:39.245596    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:39.258580 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:39.258537    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="unmounted volumes=[metrics-certs], unattached volumes=[], failed to process volumes=[]: context deadline exceeded" pod="openshift-multus/network-metrics-daemon-mlpsc" podUID="90e9cadb-328a-47e1-a119-bae6987ac229"
May 28 08:14:39.371567 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:39.371538    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-image-registry/image-registry-798b45b98d-ctqd2"]
May 28 08:14:39.377289 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:14:39.375407    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfa061e87_5cd2_4e3c_b6b0_eed405427b1c.slice/crio-21d41a438c481b0348e4ce4656d8c90ef39819a7d8f9813f71eedf1d4952cceb WatchSource:0}: Error finding container 21d41a438c481b0348e4ce4656d8c90ef39819a7d8f9813f71eedf1d4952cceb: Status 404 returned error can't find the container with id 21d41a438c481b0348e4ce4656d8c90ef39819a7d8f9813f71eedf1d4952cceb
May 28 08:14:39.715068 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:39.714983    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2" event={"ID":"fa061e87-5cd2-4e3c-b6b0-eed405427b1c","Type":"ContainerStarted","Data":"de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96"}
May 28 08:14:39.715068 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:39.715019    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2" event={"ID":"fa061e87-5cd2-4e3c-b6b0-eed405427b1c","Type":"ContainerStarted","Data":"21d41a438c481b0348e4ce4656d8c90ef39819a7d8f9813f71eedf1d4952cceb"}
May 28 08:14:39.715467 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:39.715115    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:14:39.735794 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:39.735733    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2" podStartSLOduration=16.735711396 podStartE2EDuration="16.735711396s" podCreationTimestamp="2026-05-28 08:14:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:14:39.734838929 +0000 UTC m=+158.171018094" watchObservedRunningTime="2026-05-28 08:14:39.735711396 +0000 UTC m=+158.171890563"
May 28 08:14:43.339429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:43.339384    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:14:43.339968 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:43.339479    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:14:43.341882 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:43.341859    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-tls\" (UniqueName: \"kubernetes.io/secret/48d7931c-9e47-424d-aaaf-1d3c90add99a-metrics-tls\") pod \"dns-default-vcwq4\" (UID: \"48d7931c-9e47-424d-aaaf-1d3c90add99a\") " pod="openshift-dns/dns-default-vcwq4"
May 28 08:14:43.341979 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:43.341892    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cert\" (UniqueName: \"kubernetes.io/secret/50981871-e522-44dc-b0a5-209d7ec9958f-cert\") pod \"ingress-canary-sjzdf\" (UID: \"50981871-e522-44dc-b0a5-209d7ec9958f\") " pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:14:43.516499 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:43.516464    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-dns\"/\"dns-dockercfg-22w5z\""
May 28 08:14:43.522627 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:43.522602    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-dns/dns-default-vcwq4"
May 28 08:14:43.674929 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:43.674899    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-dns/dns-default-vcwq4"]
May 28 08:14:43.677340 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:14:43.677311    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod48d7931c_9e47_424d_aaaf_1d3c90add99a.slice/crio-8d301d9227a6c4e9959d34251dca40df23cde7c7e1dc6c2341b721cc5fa48c40 WatchSource:0}: Error finding container 8d301d9227a6c4e9959d34251dca40df23cde7c7e1dc6c2341b721cc5fa48c40: Status 404 returned error can't find the container with id 8d301d9227a6c4e9959d34251dca40df23cde7c7e1dc6c2341b721cc5fa48c40
May 28 08:14:43.726491 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:43.726457    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vcwq4" event={"ID":"48d7931c-9e47-424d-aaaf-1d3c90add99a","Type":"ContainerStarted","Data":"8d301d9227a6c4e9959d34251dca40df23cde7c7e1dc6c2341b721cc5fa48c40"}
May 28 08:14:45.733749 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:45.733716    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vcwq4" event={"ID":"48d7931c-9e47-424d-aaaf-1d3c90add99a","Type":"ContainerStarted","Data":"2d08a5747f7aac63b58b19d0bbf25a3438b71b08ddc0d5d6dc8899243d4d9730"}
May 28 08:14:45.733749 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:45.733751    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-dns/dns-default-vcwq4" event={"ID":"48d7931c-9e47-424d-aaaf-1d3c90add99a","Type":"ContainerStarted","Data":"b878ed2bff792d600725b0d28cb646626d218f254819d9a4d310dc847164ed23"}
May 28 08:14:45.734164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:45.733867    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-dns/dns-default-vcwq4"
May 28 08:14:45.752731 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:45.752682    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-dns/dns-default-vcwq4" podStartSLOduration=129.45239853 podStartE2EDuration="2m10.752646601s" podCreationTimestamp="2026-05-28 08:12:35 +0000 UTC" firstStartedPulling="2026-05-28 08:14:43.679284186 +0000 UTC m=+162.115463345" lastFinishedPulling="2026-05-28 08:14:44.979532267 +0000 UTC m=+163.415711416" observedRunningTime="2026-05-28 08:14:45.75243993 +0000 UTC m=+164.188619096" watchObservedRunningTime="2026-05-28 08:14:45.752646601 +0000 UTC m=+164.188825766"
May 28 08:14:47.237562 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:47.237532    2577 scope.go:117] "RemoveContainer" containerID="32c676400f3e12dc22cc1226c402f4bd42bc6aa0101c5b4e03ad39480a904281"
May 28 08:14:47.739237 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:47.739210    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:14:47.739613 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:47.739598    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/1.log"
May 28 08:14:47.739709 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:47.739631    2577 generic.go:358] "Generic (PLEG): container finished" podID="601ae44f-2916-47d8-9538-61dd910c49d7" containerID="2458b10334b184e2a16a6265f662c035c0a30a32c49544dd4e39e27f84bf485f" exitCode=255
May 28 08:14:47.739709 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:47.739692    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" event={"ID":"601ae44f-2916-47d8-9538-61dd910c49d7","Type":"ContainerDied","Data":"2458b10334b184e2a16a6265f662c035c0a30a32c49544dd4e39e27f84bf485f"}
May 28 08:14:47.739774 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:47.739733    2577 scope.go:117] "RemoveContainer" containerID="32c676400f3e12dc22cc1226c402f4bd42bc6aa0101c5b4e03ad39480a904281"
May 28 08:14:47.740083 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:47.740063    2577 scope.go:117] "RemoveContainer" containerID="2458b10334b184e2a16a6265f662c035c0a30a32c49544dd4e39e27f84bf485f"
May 28 08:14:47.740262 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:47.740244    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-xkwsz_openshift-console-operator(601ae44f-2916-47d8-9538-61dd910c49d7)\"" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" podUID="601ae44f-2916-47d8-9538-61dd910c49d7"
May 28 08:14:48.743497 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:48.743470    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:14:49.237964 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:49.237928    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:14:49.241158 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:49.241141    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-ingress-canary\"/\"default-dockercfg-cs8td\""
May 28 08:14:49.248832 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:49.248811    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress-canary/ingress-canary-sjzdf"
May 28 08:14:49.374527 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:49.374387    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress-canary/ingress-canary-sjzdf"]
May 28 08:14:49.376940 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:14:49.376913    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod50981871_e522_44dc_b0a5_209d7ec9958f.slice/crio-8f03282e5f8e02881bdf49d0bd3b813ac5691fe8f91cfc4adf877eaaddbb2e22 WatchSource:0}: Error finding container 8f03282e5f8e02881bdf49d0bd3b813ac5691fe8f91cfc4adf877eaaddbb2e22: Status 404 returned error can't find the container with id 8f03282e5f8e02881bdf49d0bd3b813ac5691fe8f91cfc4adf877eaaddbb2e22
May 28 08:14:49.747183 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:49.747144    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sjzdf" event={"ID":"50981871-e522-44dc-b0a5-209d7ec9958f","Type":"ContainerStarted","Data":"8f03282e5f8e02881bdf49d0bd3b813ac5691fe8f91cfc4adf877eaaddbb2e22"}
May 28 08:14:51.237939 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:51.237851    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:14:51.756908 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:51.756872    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress-canary/ingress-canary-sjzdf" event={"ID":"50981871-e522-44dc-b0a5-209d7ec9958f","Type":"ContainerStarted","Data":"a982deb5b98b5364a04a7ffe5f40df155d1a7cf38670634f5da5764e1c396f07"}
May 28 08:14:51.775411 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:51.775366    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress-canary/ingress-canary-sjzdf" podStartSLOduration=135.327588913 podStartE2EDuration="2m16.775350951s" podCreationTimestamp="2026-05-28 08:12:35 +0000 UTC" firstStartedPulling="2026-05-28 08:14:49.378756839 +0000 UTC m=+167.814935998" lastFinishedPulling="2026-05-28 08:14:50.826518889 +0000 UTC m=+169.262698036" observedRunningTime="2026-05-28 08:14:51.775087936 +0000 UTC m=+170.211267102" watchObservedRunningTime="2026-05-28 08:14:51.775350951 +0000 UTC m=+170.211530119"
May 28 08:14:53.546235 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:53.546183    2577 kubelet.go:2658] "SyncLoop (probe)" probe="liveness" status="unhealthy" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:53.546623 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:53.546537    2577 scope.go:117] "RemoveContainer" containerID="2458b10334b184e2a16a6265f662c035c0a30a32c49544dd4e39e27f84bf485f"
May 28 08:14:53.546745 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:53.546725    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-xkwsz_openshift-console-operator(601ae44f-2916-47d8-9538-61dd910c49d7)\"" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" podUID="601ae44f-2916-47d8-9538-61dd910c49d7"
May 28 08:14:54.810757 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:54.810726    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-798b45b98d-ctqd2"]
May 28 08:14:54.815341 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:54.815293    2577 patch_prober.go:28] interesting pod/image-registry-798b45b98d-ctqd2 container/registry namespace/openshift-image-registry: Readiness probe status=failure output="HTTP probe failed with statuscode: 503" start-of-body={"errors":[{"code":"UNAVAILABLE","message":"service unavailable","detail":"health check failed: please see /debug/health"}]}
May 28 08:14:54.815720 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:54.815350    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2" podUID="fa061e87-5cd2-4e3c-b6b0-eed405427b1c" containerName="registry" probeResult="failure" output="HTTP probe failed with statuscode: 503"
May 28 08:14:54.856780 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:54.856750    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-insights/insights-runtime-extractor-v8cmf"]
May 28 08:14:54.860208 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:54.860170    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:54.863081 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:54.863056    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-tls\""
May 28 08:14:54.863232 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:54.863157    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-root-ca.crt\""
May 28 08:14:54.863315 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:54.863241    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"kube-rbac-proxy\""
May 28 08:14:54.863638 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:54.863624    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-insights\"/\"insights-runtime-extractor-sa-dockercfg-hrwcb\""
May 28 08:14:54.864053 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:54.864038    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-insights\"/\"openshift-service-ca.crt\""
May 28 08:14:54.887808 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:54.887776    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v8cmf"]
May 28 08:14:55.029351 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.029308    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3fc18f79-4b6c-47d7-ad7f-65341185ec83-crio-socket\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.029513 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.029371    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:55.029513 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.029399    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3fc18f79-4b6c-47d7-ad7f-65341185ec83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.029513 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.029447    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3fc18f79-4b6c-47d7-ad7f-65341185ec83-data-volume\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.029687 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.029588    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dwg4m\" (UniqueName: \"kubernetes.io/projected/3fc18f79-4b6c-47d7-ad7f-65341185ec83-kube-api-access-dwg4m\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.029687 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.029638    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3fc18f79-4b6c-47d7-ad7f-65341185ec83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.030086 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.030053    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/2b55796c-941d-42d5-a985-40d51c23e675-service-ca-bundle\") pod \"router-default-7df5fc48dd-7nhw9\" (UID: \"2b55796c-941d-42d5-a985-40d51c23e675\") " pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:55.040604 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.040583    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:55.130139 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.130108    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3fc18f79-4b6c-47d7-ad7f-65341185ec83-data-volume\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.130304 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.130164    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dwg4m\" (UniqueName: \"kubernetes.io/projected/3fc18f79-4b6c-47d7-ad7f-65341185ec83-kube-api-access-dwg4m\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.130304 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.130195    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3fc18f79-4b6c-47d7-ad7f-65341185ec83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.130304 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.130228    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3fc18f79-4b6c-47d7-ad7f-65341185ec83-crio-socket\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.130304 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.130266    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3fc18f79-4b6c-47d7-ad7f-65341185ec83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.130506 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.130404    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"crio-socket\" (UniqueName: \"kubernetes.io/host-path/3fc18f79-4b6c-47d7-ad7f-65341185ec83-crio-socket\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.130561 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.130521    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"data-volume\" (UniqueName: \"kubernetes.io/empty-dir/3fc18f79-4b6c-47d7-ad7f-65341185ec83-data-volume\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.130735 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.130719    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-rbac-proxy-cm\" (UniqueName: \"kubernetes.io/configmap/3fc18f79-4b6c-47d7-ad7f-65341185ec83-kube-rbac-proxy-cm\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.132650 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.132612    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"insights-runtime-extractor-tls\" (UniqueName: \"kubernetes.io/secret/3fc18f79-4b6c-47d7-ad7f-65341185ec83-insights-runtime-extractor-tls\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.140990 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.140963    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dwg4m\" (UniqueName: \"kubernetes.io/projected/3fc18f79-4b6c-47d7-ad7f-65341185ec83-kube-api-access-dwg4m\") pod \"insights-runtime-extractor-v8cmf\" (UID: \"3fc18f79-4b6c-47d7-ad7f-65341185ec83\") " pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.170603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.170020    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-insights/insights-runtime-extractor-v8cmf"
May 28 08:14:55.172536 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.172515    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-ingress/router-default-7df5fc48dd-7nhw9"]
May 28 08:14:55.177884 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:14:55.177844    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2b55796c_941d_42d5_a985_40d51c23e675.slice/crio-ae6f394f20ddebd917a0a656e36b3e92aeffda7ee979dcd02b573fd585d9bf0a WatchSource:0}: Error finding container ae6f394f20ddebd917a0a656e36b3e92aeffda7ee979dcd02b573fd585d9bf0a: Status 404 returned error can't find the container with id ae6f394f20ddebd917a0a656e36b3e92aeffda7ee979dcd02b573fd585d9bf0a
May 28 08:14:55.318019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.317993    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-insights/insights-runtime-extractor-v8cmf"]
May 28 08:14:55.320513 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:14:55.320476    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod3fc18f79_4b6c_47d7_ad7f_65341185ec83.slice/crio-ed3d79ced5c7bb51b6f7957bf1d9cb2c02a8a57ef0c70da1da8e0cbe1f559da8 WatchSource:0}: Error finding container ed3d79ced5c7bb51b6f7957bf1d9cb2c02a8a57ef0c70da1da8e0cbe1f559da8: Status 404 returned error can't find the container with id ed3d79ced5c7bb51b6f7957bf1d9cb2c02a8a57ef0c70da1da8e0cbe1f559da8
May 28 08:14:55.674360 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.674270    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:14:55.674686 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.674646    2577 scope.go:117] "RemoveContainer" containerID="2458b10334b184e2a16a6265f662c035c0a30a32c49544dd4e39e27f84bf485f"
May 28 08:14:55.674867 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:14:55.674850    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-xkwsz_openshift-console-operator(601ae44f-2916-47d8-9538-61dd910c49d7)\"" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" podUID="601ae44f-2916-47d8-9538-61dd910c49d7"
May 28 08:14:55.738348 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.738317    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-dns/dns-default-vcwq4"
May 28 08:14:55.770533 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.770488    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7df5fc48dd-7nhw9" event={"ID":"2b55796c-941d-42d5-a985-40d51c23e675","Type":"ContainerStarted","Data":"4f30ed90289cf2fd4e684361787d12c6c74739b00ac31b4c201d138c886d6d74"}
May 28 08:14:55.770533 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.770537    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-ingress/router-default-7df5fc48dd-7nhw9" event={"ID":"2b55796c-941d-42d5-a985-40d51c23e675","Type":"ContainerStarted","Data":"ae6f394f20ddebd917a0a656e36b3e92aeffda7ee979dcd02b573fd585d9bf0a"}
May 28 08:14:55.771709 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.771684    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8cmf" event={"ID":"3fc18f79-4b6c-47d7-ad7f-65341185ec83","Type":"ContainerStarted","Data":"01c240cd90d9d9dc78471eedfb4d682a6864b6224b48ac81cdd4315cbc2cf215"}
May 28 08:14:55.771709 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.771712    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8cmf" event={"ID":"3fc18f79-4b6c-47d7-ad7f-65341185ec83","Type":"ContainerStarted","Data":"ed3d79ced5c7bb51b6f7957bf1d9cb2c02a8a57ef0c70da1da8e0cbe1f559da8"}
May 28 08:14:55.807925 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:55.807871    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-ingress/router-default-7df5fc48dd-7nhw9" podStartSLOduration=32.807855069 podStartE2EDuration="32.807855069s" podCreationTimestamp="2026-05-28 08:14:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:14:55.806570778 +0000 UTC m=+174.242749944" watchObservedRunningTime="2026-05-28 08:14:55.807855069 +0000 UTC m=+174.244034234"
May 28 08:14:56.041586 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:56.041544    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:56.044311 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:56.044288    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:56.778102 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:56.778059    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8cmf" event={"ID":"3fc18f79-4b6c-47d7-ad7f-65341185ec83","Type":"ContainerStarted","Data":"b53bf1fcaa8cbe7affdd647eb014033238edbe221dcbf149fc751902b2780ace"}
May 28 08:14:56.778402 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:56.778380    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:56.779840 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:56.779814    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-ingress/router-default-7df5fc48dd-7nhw9"
May 28 08:14:57.782505 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:57.782469    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-insights/insights-runtime-extractor-v8cmf" event={"ID":"3fc18f79-4b6c-47d7-ad7f-65341185ec83","Type":"ContainerStarted","Data":"9230e716b55fc5b7f4c929cc30d7d4cfbbaa93c3452d1732e3807d942e1085d5"}
May 28 08:14:57.802572 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:14:57.802528    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-insights/insights-runtime-extractor-v8cmf" podStartSLOduration=1.911204123 podStartE2EDuration="3.802516177s" podCreationTimestamp="2026-05-28 08:14:54 +0000 UTC" firstStartedPulling="2026-05-28 08:14:55.375267619 +0000 UTC m=+173.811446763" lastFinishedPulling="2026-05-28 08:14:57.26657967 +0000 UTC m=+175.702758817" observedRunningTime="2026-05-28 08:14:57.800684538 +0000 UTC m=+176.236863704" watchObservedRunningTime="2026-05-28 08:14:57.802516177 +0000 UTC m=+176.238695343"
May 28 08:15:04.815350 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:04.815321    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:15:05.732066 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.732027    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/node-exporter-vx4q6"]
May 28 08:15:05.736484 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.736458    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"]
May 28 08:15:05.736681 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.736645    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.741094 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.741057    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"metrics-client-ca\""
May 28 08:15:05.741367 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.741350    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-dockercfg-5v6rr\""
May 28 08:15:05.741700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.741580    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-tls\""
May 28 08:15:05.745942 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.745913    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"node-exporter-accelerators-collector-config\""
May 28 08:15:05.746368 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.746216    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"node-exporter-kube-rbac-proxy-config\""
May 28 08:15:05.746517 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.746495    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-root-ca.crt\""
May 28 08:15:05.748112 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.746864    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"openshift-service-ca.crt\""
May 28 08:15:05.748112 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.747380    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.749602 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.749578    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"]
May 28 08:15:05.750380 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.750192    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-tls\""
May 28 08:15:05.750951 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.750932    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-custom-resource-state-configmap\""
May 28 08:15:05.752725 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.752707    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-dockercfg-bhs4t\""
May 28 08:15:05.753135 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.753117    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"kube-state-metrics-kube-rbac-proxy-config\""
May 28 08:15:05.808831 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.808790    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-wtmp\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.809028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.808891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ceab5153-c91d-46fc-89ba-a795bd986c0e-metrics-client-ca\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.809028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.808922    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.809028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.808959    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.809028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.808999    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-tls\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.809028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.809026    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ceab5153-c91d-46fc-89ba-a795bd986c0e-root\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.809283 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.809053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-textfile\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.809283 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.809092    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.809283 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.809122    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8rj84\" (UniqueName: \"kubernetes.io/projected/ceab5153-c91d-46fc-89ba-a795bd986c0e-kube-api-access-8rj84\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.809283 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.809149    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bb38ee1-d552-4e83-bb65-7ab76b50e910-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.809283 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.809172    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ceab5153-c91d-46fc-89ba-a795bd986c0e-sys\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.809283 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.809196    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.809283 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.809237    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-accelerators-collector-config\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.809283 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.809261    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1bb38ee1-d552-4e83-bb65-7ab76b50e910-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.809697 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.809353    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lsrlk\" (UniqueName: \"kubernetes.io/projected/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-api-access-lsrlk\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.910233 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910188    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-accelerators-collector-config\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.910233 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910235    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1bb38ee1-d552-4e83-bb65-7ab76b50e910-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910290    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lsrlk\" (UniqueName: \"kubernetes.io/projected/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-api-access-lsrlk\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910317    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-wtmp\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910351    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ceab5153-c91d-46fc-89ba-a795bd986c0e-metrics-client-ca\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910375    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910404    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910446    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-tls\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910473    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ceab5153-c91d-46fc-89ba-a795bd986c0e-root\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910504    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-textfile\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910547    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910575    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8rj84\" (UniqueName: \"kubernetes.io/projected/ceab5153-c91d-46fc-89ba-a795bd986c0e-kube-api-access-8rj84\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910599    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bb38ee1-d552-4e83-bb65-7ab76b50e910-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910620    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ceab5153-c91d-46fc-89ba-a795bd986c0e-sys\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.910806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.910642    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.911731 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:05.911703    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 28 08:15:05.912857 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:05.912835    2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
May 28 08:15:05.913124 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.912960    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"volume-directive-shadow\" (UniqueName: \"kubernetes.io/empty-dir/1bb38ee1-d552-4e83-bb65-7ab76b50e910-volume-directive-shadow\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.913124 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.912280    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-wtmp\" (UniqueName: \"kubernetes.io/host-path/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-wtmp\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.913124 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.912421    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/1bb38ee1-d552-4e83-bb65-7ab76b50e910-metrics-client-ca\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.913124 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.912736    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-accelerators-collector-config\" (UniqueName: \"kubernetes.io/configmap/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-accelerators-collector-config\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.913124 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.912759    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/ceab5153-c91d-46fc-89ba-a795bd986c0e-metrics-client-ca\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.913124 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.912789    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/ceab5153-c91d-46fc-89ba-a795bd986c0e-sys\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.913124 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.911891    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-textfile\" (UniqueName: \"kubernetes.io/empty-dir/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-textfile\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.913124 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.912147    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-custom-resource-state-configmap\" (UniqueName: \"kubernetes.io/configmap/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-custom-resource-state-configmap\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.913124 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:05.912914    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-tls podName:ceab5153-c91d-46fc-89ba-a795bd986c0e nodeName:}" failed. No retries permitted until 2026-05-28 08:15:06.41286324 +0000 UTC m=+184.849042391 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-tls") pod "node-exporter-vx4q6" (UID: "ceab5153-c91d-46fc-89ba-a795bd986c0e") : secret "node-exporter-tls" not found
May 28 08:15:05.913124 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:05.913096    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-tls podName:1bb38ee1-d552-4e83-bb65-7ab76b50e910 nodeName:}" failed. No retries permitted until 2026-05-28 08:15:06.413076494 +0000 UTC m=+184.849255641 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-tls") pod "kube-state-metrics-54fd6bc5c4-c8cdd" (UID: "1bb38ee1-d552-4e83-bb65-7ab76b50e910") : secret "kube-state-metrics-tls" not found
May 28 08:15:05.915348 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.915209    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"root\" (UniqueName: \"kubernetes.io/host-path/ceab5153-c91d-46fc-89ba-a795bd986c0e-root\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.915425 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.915397    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-kube-rbac-proxy-config\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:05.915714 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.915693    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-kube-rbac-proxy-config\" (UniqueName: \"kubernetes.io/secret/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-kube-rbac-proxy-config\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.922996 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.922938    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lsrlk\" (UniqueName: \"kubernetes.io/projected/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-api-access-lsrlk\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:05.923684 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:05.923640    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8rj84\" (UniqueName: \"kubernetes.io/projected/ceab5153-c91d-46fc-89ba-a795bd986c0e-kube-api-access-8rj84\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:06.238021 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.237987    2577 scope.go:117] "RemoveContainer" containerID="2458b10334b184e2a16a6265f662c035c0a30a32c49544dd4e39e27f84bf485f"
May 28 08:15:06.238276 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:06.238248    2577 pod_workers.go:1301] "Error syncing pod, skipping" err="failed to \"StartContainer\" for \"console-operator\" with CrashLoopBackOff: \"back-off 20s restarting failed container=console-operator pod=console-operator-76dff48d59-xkwsz_openshift-console-operator(601ae44f-2916-47d8-9538-61dd910c49d7)\"" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" podUID="601ae44f-2916-47d8-9538-61dd910c49d7"
May 28 08:15:06.415768 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.415719    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-tls\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:06.415977 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.415857    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:06.415977 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:06.415868    2577 secret.go:189] Couldn't get secret openshift-monitoring/node-exporter-tls: secret "node-exporter-tls" not found
May 28 08:15:06.415977 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:06.415930    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-tls podName:ceab5153-c91d-46fc-89ba-a795bd986c0e nodeName:}" failed. No retries permitted until 2026-05-28 08:15:07.415913475 +0000 UTC m=+185.852092622 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "node-exporter-tls" (UniqueName: "kubernetes.io/secret/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-tls") pod "node-exporter-vx4q6" (UID: "ceab5153-c91d-46fc-89ba-a795bd986c0e") : secret "node-exporter-tls" not found
May 28 08:15:06.415977 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:06.415974    2577 secret.go:189] Couldn't get secret openshift-monitoring/kube-state-metrics-tls: secret "kube-state-metrics-tls" not found
May 28 08:15:06.416188 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:06.416040    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-tls podName:1bb38ee1-d552-4e83-bb65-7ab76b50e910 nodeName:}" failed. No retries permitted until 2026-05-28 08:15:07.416023614 +0000 UTC m=+185.852202764 (durationBeforeRetry 1s). Error: MountVolume.SetUp failed for volume "kube-state-metrics-tls" (UniqueName: "kubernetes.io/secret/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-tls") pod "kube-state-metrics-54fd6bc5c4-c8cdd" (UID: "1bb38ee1-d552-4e83-bb65-7ab76b50e910") : secret "kube-state-metrics-tls" not found
May 28 08:15:06.821203 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.821163    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 08:15:06.826691 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.826639    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.829781 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.829746    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 28 08:15:06.829940 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.829809    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 28 08:15:06.829940 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.829833    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 28 08:15:06.829940 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.829898    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 28 08:15:06.829940 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.829936    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 28 08:15:06.830184 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.829971    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 28 08:15:06.830184 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.830032    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 28 08:15:06.830280 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.830208    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 28 08:15:06.830280 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.830239    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hxcjg\""
May 28 08:15:06.830361 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.830240    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 28 08:15:06.837509 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.837485    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 08:15:06.919683 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.919615    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.920131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.919692    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-config-volume\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.920131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.919716    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/faf2d625-97c7-48b9-826b-567b708ee418-tls-assets\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.920131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.919737    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/faf2d625-97c7-48b9-826b-567b708ee418-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.920131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.919761    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/faf2d625-97c7-48b9-826b-567b708ee418-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.920131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.919897    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faf2d625-97c7-48b9-826b-567b708ee418-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.920131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.919946    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-web-config\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.920131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.919980    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8nnn9\" (UniqueName: \"kubernetes.io/projected/faf2d625-97c7-48b9-826b-567b708ee418-kube-api-access-8nnn9\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.920131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.920028    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.920131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.920053    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/faf2d625-97c7-48b9-826b-567b708ee418-config-out\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.920131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.920069    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.920131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.920096    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:06.920131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:06.920136    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.020563 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020520    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faf2d625-97c7-48b9-826b-567b708ee418-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.020563 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020566    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-web-config\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.020795 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020583    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8nnn9\" (UniqueName: \"kubernetes.io/projected/faf2d625-97c7-48b9-826b-567b708ee418-kube-api-access-8nnn9\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.020795 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020616    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.020795 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020646    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/faf2d625-97c7-48b9-826b-567b708ee418-config-out\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.020795 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020688    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.020795 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020718    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.020795 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020760    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.021064 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020808    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.021064 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020839    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-config-volume\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.021064 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020860    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/faf2d625-97c7-48b9-826b-567b708ee418-tls-assets\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.021064 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020922    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/faf2d625-97c7-48b9-826b-567b708ee418-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.021064 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.020984    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/faf2d625-97c7-48b9-826b-567b708ee418-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.021719 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.021370    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/faf2d625-97c7-48b9-826b-567b708ee418-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.021719 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.021559    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faf2d625-97c7-48b9-826b-567b708ee418-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.022164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.022141    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/faf2d625-97c7-48b9-826b-567b708ee418-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.024220 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.024137    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.024220 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.024141    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.024220 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.024190    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-web-config\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.024434 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:07.024248    2577 secret.go:189] Couldn't get secret openshift-monitoring/alertmanager-main-tls: secret "alertmanager-main-tls" not found
May 28 08:15:07.024434 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:07.024338    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-main-tls podName:faf2d625-97c7-48b9-826b-567b708ee418 nodeName:}" failed. No retries permitted until 2026-05-28 08:15:07.524315998 +0000 UTC m=+185.960495160 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "secret-alertmanager-main-tls" (UniqueName: "kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-main-tls") pod "alertmanager-main-0" (UID: "faf2d625-97c7-48b9-826b-567b708ee418") : secret "alertmanager-main-tls" not found
May 28 08:15:07.024537 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.024454    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-config-volume\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.024897 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.024867    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.025016 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.024972    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.025016 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.025006    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/faf2d625-97c7-48b9-826b-567b708ee418-tls-assets\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.025460 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.025445    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/faf2d625-97c7-48b9-826b-567b708ee418-config-out\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.029549 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.029522    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8nnn9\" (UniqueName: \"kubernetes.io/projected/faf2d625-97c7-48b9-826b-567b708ee418-kube-api-access-8nnn9\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.424633 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.424590    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-tls\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:07.424834 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.424717    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:07.427162 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.427135    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"node-exporter-tls\" (UniqueName: \"kubernetes.io/secret/ceab5153-c91d-46fc-89ba-a795bd986c0e-node-exporter-tls\") pod \"node-exporter-vx4q6\" (UID: \"ceab5153-c91d-46fc-89ba-a795bd986c0e\") " pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:07.427288 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.427274    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-state-metrics-tls\" (UniqueName: \"kubernetes.io/secret/1bb38ee1-d552-4e83-bb65-7ab76b50e910-kube-state-metrics-tls\") pod \"kube-state-metrics-54fd6bc5c4-c8cdd\" (UID: \"1bb38ee1-d552-4e83-bb65-7ab76b50e910\") " pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:07.525260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.525222    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.527752 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.527730    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.555609 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.555572    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/node-exporter-vx4q6"
May 28 08:15:07.561447 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.561424    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"
May 28 08:15:07.564331 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:15:07.564302    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podceab5153_c91d_46fc_89ba_a795bd986c0e.slice/crio-7d92462aa5b329648134d7e82dc8537719ce683a8ca4dad0be91971d8c829664 WatchSource:0}: Error finding container 7d92462aa5b329648134d7e82dc8537719ce683a8ca4dad0be91971d8c829664: Status 404 returned error can't find the container with id 7d92462aa5b329648134d7e82dc8537719ce683a8ca4dad0be91971d8c829664
May 28 08:15:07.699907 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.699885    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd"]
May 28 08:15:07.701585 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:15:07.701555    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod1bb38ee1_d552_4e83_bb65_7ab76b50e910.slice/crio-95ca397ccf70ce6625dc9bf6c1ac4d754b00a4da3a777b4c88877d334f078156 WatchSource:0}: Error finding container 95ca397ccf70ce6625dc9bf6c1ac4d754b00a4da3a777b4c88877d334f078156: Status 404 returned error can't find the container with id 95ca397ccf70ce6625dc9bf6c1ac4d754b00a4da3a777b4c88877d334f078156
May 28 08:15:07.736717 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.736669    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 08:15:07.807716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.807680    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vx4q6" event={"ID":"ceab5153-c91d-46fc-89ba-a795bd986c0e","Type":"ContainerStarted","Data":"7d92462aa5b329648134d7e82dc8537719ce683a8ca4dad0be91971d8c829664"}
May 28 08:15:07.808830 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.808800    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd" event={"ID":"1bb38ee1-d552-4e83-bb65-7ab76b50e910","Type":"ContainerStarted","Data":"95ca397ccf70ce6625dc9bf6c1ac4d754b00a4da3a777b4c88877d334f078156"}
May 28 08:15:07.878402 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:07.878304    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 08:15:07.880745 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:15:07.880719    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podfaf2d625_97c7_48b9_826b_567b708ee418.slice/crio-ab1371f7ad3f3ef09e798fc5a335af53841787b5191370a3c6bbb9c6150032f5 WatchSource:0}: Error finding container ab1371f7ad3f3ef09e798fc5a335af53841787b5191370a3c6bbb9c6150032f5: Status 404 returned error can't find the container with id ab1371f7ad3f3ef09e798fc5a335af53841787b5191370a3c6bbb9c6150032f5
May 28 08:15:08.811371 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.811335    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/thanos-querier-869899b58f-86bv9"]
May 28 08:15:08.815272 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.815244    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerStarted","Data":"ab1371f7ad3f3ef09e798fc5a335af53841787b5191370a3c6bbb9c6150032f5"}
May 28 08:15:08.815421 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.815397    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.818722 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.818692    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-rules\""
May 28 08:15:08.818958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.818939    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-web\""
May 28 08:15:08.819032 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.818939    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-grpc-tls-fosgkgjig3bui\""
May 28 08:15:08.819148 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.819131    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy-metrics\""
May 28 08:15:08.819430 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.819409    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-kube-rbac-proxy\""
May 28 08:15:08.819629 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.819608    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-dockercfg-7rnk8\""
May 28 08:15:08.819743 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.819630    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"thanos-querier-tls\""
May 28 08:15:08.831378 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.831337    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-869899b58f-86bv9"]
May 28 08:15:08.835774 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.835739    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.835918 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.835825    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.835918 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.835855    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.835918 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.835876    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-grpc-tls\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.836086 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.835930    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j6j8t\" (UniqueName: \"kubernetes.io/projected/5a6bfad0-ca17-41bb-995f-73538d18a6c5-kube-api-access-j6j8t\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.836086 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.835957    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.836086 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.836027    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-tls\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.836086 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.836054    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a6bfad0-ca17-41bb-995f-73538d18a6c5-metrics-client-ca\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.937481 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.937437    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j6j8t\" (UniqueName: \"kubernetes.io/projected/5a6bfad0-ca17-41bb-995f-73538d18a6c5-kube-api-access-j6j8t\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.937481 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.937488    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.937739 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.937521    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-tls\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.937739 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.937544    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a6bfad0-ca17-41bb-995f-73538d18a6c5-metrics-client-ca\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.937739 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.937584    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.937739 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.937649    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.937739 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.937700    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.937739 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.937724    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-grpc-tls\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.938972 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.938940    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/5a6bfad0-ca17-41bb-995f-73538d18a6c5-metrics-client-ca\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.940793 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.940763    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-metrics\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-kube-rbac-proxy-metrics\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.941083 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.941016    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-rules\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-kube-rbac-proxy-rules\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.941224 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.941187    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-kube-rbac-proxy-web\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.941571 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.941526    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-grpc-tls\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-grpc-tls\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.942336 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.942289    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-kube-rbac-proxy\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.943606 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.943585    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-thanos-querier-tls\" (UniqueName: \"kubernetes.io/secret/5a6bfad0-ca17-41bb-995f-73538d18a6c5-secret-thanos-querier-tls\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:08.948790 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:08.948769    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j6j8t\" (UniqueName: \"kubernetes.io/projected/5a6bfad0-ca17-41bb-995f-73538d18a6c5-kube-api-access-j6j8t\") pod \"thanos-querier-869899b58f-86bv9\" (UID: \"5a6bfad0-ca17-41bb-995f-73538d18a6c5\") " pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:09.127475 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:09.127361    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:09.384020 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:09.383988    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/thanos-querier-869899b58f-86bv9"]
May 28 08:15:09.415695 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:15:09.415630    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5a6bfad0_ca17_41bb_995f_73538d18a6c5.slice/crio-2bd5b8ec1d360ac6d364cd3b9625ebc10d6fcbdb5ab45105edc17000f310cbb9 WatchSource:0}: Error finding container 2bd5b8ec1d360ac6d364cd3b9625ebc10d6fcbdb5ab45105edc17000f310cbb9: Status 404 returned error can't find the container with id 2bd5b8ec1d360ac6d364cd3b9625ebc10d6fcbdb5ab45105edc17000f310cbb9
May 28 08:15:09.817504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:09.817465    2577 generic.go:358] "Generic (PLEG): container finished" podID="faf2d625-97c7-48b9-826b-567b708ee418" containerID="01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485" exitCode=0
May 28 08:15:09.817981 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:09.817539    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerDied","Data":"01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485"}
May 28 08:15:09.818708 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:09.818681    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-869899b58f-86bv9" event={"ID":"5a6bfad0-ca17-41bb-995f-73538d18a6c5","Type":"ContainerStarted","Data":"2bd5b8ec1d360ac6d364cd3b9625ebc10d6fcbdb5ab45105edc17000f310cbb9"}
May 28 08:15:09.820507 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:09.820486    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd" event={"ID":"1bb38ee1-d552-4e83-bb65-7ab76b50e910","Type":"ContainerStarted","Data":"3acc43963c4e1385301c37872e34b7dc90436addedd0e0352fd26829a69fcc66"}
May 28 08:15:09.820604 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:09.820513    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd" event={"ID":"1bb38ee1-d552-4e83-bb65-7ab76b50e910","Type":"ContainerStarted","Data":"ba338d54f2086178b1b0257d5aa46b71b47006fd4e482c9ee39ca7d24e2df8d0"}
May 28 08:15:09.820604 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:09.820523    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd" event={"ID":"1bb38ee1-d552-4e83-bb65-7ab76b50e910","Type":"ContainerStarted","Data":"810810f093d02c3f9da8ad114051dd8da25a0fc35f8ef3a331220e5bf5198b7f"}
May 28 08:15:09.821958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:09.821937    2577 generic.go:358] "Generic (PLEG): container finished" podID="ceab5153-c91d-46fc-89ba-a795bd986c0e" containerID="cb1a08638df37ee0ae28faf1c370bc061a007c34952169daefa510f0f00b39e5" exitCode=0
May 28 08:15:09.822052 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:09.821976    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vx4q6" event={"ID":"ceab5153-c91d-46fc-89ba-a795bd986c0e","Type":"ContainerDied","Data":"cb1a08638df37ee0ae28faf1c370bc061a007c34952169daefa510f0f00b39e5"}
May 28 08:15:09.889302 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:09.889244    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/kube-state-metrics-54fd6bc5c4-c8cdd" podStartSLOduration=3.357680383 podStartE2EDuration="4.889226499s" podCreationTimestamp="2026-05-28 08:15:05 +0000 UTC" firstStartedPulling="2026-05-28 08:15:07.703518532 +0000 UTC m=+186.139697680" lastFinishedPulling="2026-05-28 08:15:09.235064637 +0000 UTC m=+187.671243796" observedRunningTime="2026-05-28 08:15:09.88825139 +0000 UTC m=+188.324430560" watchObservedRunningTime="2026-05-28 08:15:09.889226499 +0000 UTC m=+188.325405679"
May 28 08:15:10.498423 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:10.498324    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-msdhf"]
May 28 08:15:10.511070 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:10.511026    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-msdhf"]
May 28 08:15:10.511219 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:10.511184    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-msdhf"
May 28 08:15:10.513892 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:10.513864    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"monitoring-plugin-cert\""
May 28 08:15:10.514045 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:10.514024    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"default-dockercfg-zd29p\""
May 28 08:15:10.556694 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:10.556607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f6caeea-d618-42f1-a51d-3f6c69aef481-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-msdhf\" (UID: \"5f6caeea-d618-42f1-a51d-3f6c69aef481\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-msdhf"
May 28 08:15:10.658233 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:10.658113    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f6caeea-d618-42f1-a51d-3f6c69aef481-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-msdhf\" (UID: \"5f6caeea-d618-42f1-a51d-3f6c69aef481\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-msdhf"
May 28 08:15:10.658418 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:10.658289    2577 secret.go:189] Couldn't get secret openshift-monitoring/monitoring-plugin-cert: secret "monitoring-plugin-cert" not found
May 28 08:15:10.658418 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:10.658377    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/5f6caeea-d618-42f1-a51d-3f6c69aef481-monitoring-plugin-cert podName:5f6caeea-d618-42f1-a51d-3f6c69aef481 nodeName:}" failed. No retries permitted until 2026-05-28 08:15:11.1583513 +0000 UTC m=+189.594530467 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "monitoring-plugin-cert" (UniqueName: "kubernetes.io/secret/5f6caeea-d618-42f1-a51d-3f6c69aef481-monitoring-plugin-cert") pod "monitoring-plugin-665c9c8584-msdhf" (UID: "5f6caeea-d618-42f1-a51d-3f6c69aef481") : secret "monitoring-plugin-cert" not found
May 28 08:15:10.829243 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:10.829189    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vx4q6" event={"ID":"ceab5153-c91d-46fc-89ba-a795bd986c0e","Type":"ContainerStarted","Data":"7fc6780e953527fcc6a4f6a199ab7c9620e5d7d2b80c7dfe3657bd3696990b50"}
May 28 08:15:10.829243 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:10.829239    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/node-exporter-vx4q6" event={"ID":"ceab5153-c91d-46fc-89ba-a795bd986c0e","Type":"ContainerStarted","Data":"6cca7201e83445e4d5e951bb48ed9fa824def6848b43e4d4e4628897095c3122"}
May 28 08:15:10.848988 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:10.848934    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/node-exporter-vx4q6" podStartSLOduration=4.189286118 podStartE2EDuration="5.848893843s" podCreationTimestamp="2026-05-28 08:15:05 +0000 UTC" firstStartedPulling="2026-05-28 08:15:07.566594861 +0000 UTC m=+186.002774008" lastFinishedPulling="2026-05-28 08:15:09.226202586 +0000 UTC m=+187.662381733" observedRunningTime="2026-05-28 08:15:10.848009577 +0000 UTC m=+189.284188745" watchObservedRunningTime="2026-05-28 08:15:10.848893843 +0000 UTC m=+189.285073009"
May 28 08:15:11.163521 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:11.163419    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f6caeea-d618-42f1-a51d-3f6c69aef481-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-msdhf\" (UID: \"5f6caeea-d618-42f1-a51d-3f6c69aef481\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-msdhf"
May 28 08:15:11.166593 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:11.166555    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"monitoring-plugin-cert\" (UniqueName: \"kubernetes.io/secret/5f6caeea-d618-42f1-a51d-3f6c69aef481-monitoring-plugin-cert\") pod \"monitoring-plugin-665c9c8584-msdhf\" (UID: \"5f6caeea-d618-42f1-a51d-3f6c69aef481\") " pod="openshift-monitoring/monitoring-plugin-665c9c8584-msdhf"
May 28 08:15:11.422399 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:11.422309    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/monitoring-plugin-665c9c8584-msdhf"
May 28 08:15:11.948364 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:11.948327    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/monitoring-plugin-665c9c8584-msdhf"]
May 28 08:15:11.951047 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:15:11.951001    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5f6caeea_d618_42f1_a51d_3f6c69aef481.slice/crio-3373e1bf0fc9a83047df62ad50e014d446d76e56d33a2128f90f6ae2fa58fde8 WatchSource:0}: Error finding container 3373e1bf0fc9a83047df62ad50e014d446d76e56d33a2128f90f6ae2fa58fde8: Status 404 returned error can't find the container with id 3373e1bf0fc9a83047df62ad50e014d446d76e56d33a2128f90f6ae2fa58fde8
May 28 08:15:12.837817 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:12.837775    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-869899b58f-86bv9" event={"ID":"5a6bfad0-ca17-41bb-995f-73538d18a6c5","Type":"ContainerStarted","Data":"e0726b4edd6984755e1b4dd395e31ca5165d698720c1c08049faa68e087e9601"}
May 28 08:15:12.837817 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:12.837824    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-869899b58f-86bv9" event={"ID":"5a6bfad0-ca17-41bb-995f-73538d18a6c5","Type":"ContainerStarted","Data":"81bea1b04cf962634a7b938d923ffa59e00928515adc3d641871705fc577945a"}
May 28 08:15:12.838064 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:12.837839    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-869899b58f-86bv9" event={"ID":"5a6bfad0-ca17-41bb-995f-73538d18a6c5","Type":"ContainerStarted","Data":"3a6af1d7f609d7fa9aa949ff7927449b8e25cea708fb163a22a77fefe54f62db"}
May 28 08:15:12.838977 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:12.838947    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-msdhf" event={"ID":"5f6caeea-d618-42f1-a51d-3f6c69aef481","Type":"ContainerStarted","Data":"3373e1bf0fc9a83047df62ad50e014d446d76e56d33a2128f90f6ae2fa58fde8"}
May 28 08:15:12.841880 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:12.841851    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerStarted","Data":"832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1"}
May 28 08:15:12.841974 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:12.841888    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerStarted","Data":"6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3"}
May 28 08:15:12.841974 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:12.841902    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerStarted","Data":"4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f"}
May 28 08:15:12.841974 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:12.841915    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerStarted","Data":"a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a"}
May 28 08:15:12.841974 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:12.841929    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerStarted","Data":"24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc"}
May 28 08:15:13.847387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:13.847350    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerStarted","Data":"24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc"}
May 28 08:15:13.849853 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:13.849826    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-869899b58f-86bv9" event={"ID":"5a6bfad0-ca17-41bb-995f-73538d18a6c5","Type":"ContainerStarted","Data":"fd3656b695aa43cb05ebf9f8f2a6ce16ccda1364b786c8ea7ad15e909aa651c2"}
May 28 08:15:13.849853 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:13.849856    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-869899b58f-86bv9" event={"ID":"5a6bfad0-ca17-41bb-995f-73538d18a6c5","Type":"ContainerStarted","Data":"00bf25df91f3d045ba78dee874bf5b5e424aad69465dd039ac32c0189bc534c7"}
May 28 08:15:13.850062 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:13.849870    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/thanos-querier-869899b58f-86bv9" event={"ID":"5a6bfad0-ca17-41bb-995f-73538d18a6c5","Type":"ContainerStarted","Data":"2ec8ba845587455c98a702151f02d37861f01cd5b27c22739d2995e165cbc145"}
May 28 08:15:13.850062 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:13.850004    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:13.851118 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:13.851097    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/monitoring-plugin-665c9c8584-msdhf" event={"ID":"5f6caeea-d618-42f1-a51d-3f6c69aef481","Type":"ContainerStarted","Data":"cad2972e54b31b41d6037ecf673417a6be5373aaf21a2f039a3753ba5ec8bc85"}
May 28 08:15:13.851293 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:13.851278    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-msdhf"
May 28 08:15:13.856475 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:13.856457    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/monitoring-plugin-665c9c8584-msdhf"
May 28 08:15:13.876615 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:13.876569    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.552866567 podStartE2EDuration="7.876558082s" podCreationTimestamp="2026-05-28 08:15:06 +0000 UTC" firstStartedPulling="2026-05-28 08:15:07.883135896 +0000 UTC m=+186.319315040" lastFinishedPulling="2026-05-28 08:15:13.206827402 +0000 UTC m=+191.643006555" observedRunningTime="2026-05-28 08:15:13.874059447 +0000 UTC m=+192.310238623" watchObservedRunningTime="2026-05-28 08:15:13.876558082 +0000 UTC m=+192.312737249"
May 28 08:15:13.892461 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:13.892404    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/monitoring-plugin-665c9c8584-msdhf" podStartSLOduration=2.641148217 podStartE2EDuration="3.892384532s" podCreationTimestamp="2026-05-28 08:15:10 +0000 UTC" firstStartedPulling="2026-05-28 08:15:11.953621728 +0000 UTC m=+190.389800887" lastFinishedPulling="2026-05-28 08:15:13.204858058 +0000 UTC m=+191.641037202" observedRunningTime="2026-05-28 08:15:13.891149598 +0000 UTC m=+192.327328765" watchObservedRunningTime="2026-05-28 08:15:13.892384532 +0000 UTC m=+192.328563890"
May 28 08:15:13.919736 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:13.919687    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/thanos-querier-869899b58f-86bv9" podStartSLOduration=2.132387242 podStartE2EDuration="5.919669192s" podCreationTimestamp="2026-05-28 08:15:08 +0000 UTC" firstStartedPulling="2026-05-28 08:15:09.417564748 +0000 UTC m=+187.853743908" lastFinishedPulling="2026-05-28 08:15:13.2048467 +0000 UTC m=+191.641025858" observedRunningTime="2026-05-28 08:15:13.917908852 +0000 UTC m=+192.354088033" watchObservedRunningTime="2026-05-28 08:15:13.919669192 +0000 UTC m=+192.355848356"
May 28 08:15:19.237443 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:19.237412    2577 scope.go:117] "RemoveContainer" containerID="2458b10334b184e2a16a6265f662c035c0a30a32c49544dd4e39e27f84bf485f"
May 28 08:15:19.830205 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:19.830166    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2" podUID="fa061e87-5cd2-4e3c-b6b0-eed405427b1c" containerName="registry" containerID="cri-o://de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96" gracePeriod=30
May 28 08:15:19.860034 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:19.860007    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-monitoring/thanos-querier-869899b58f-86bv9"
May 28 08:15:19.870767 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:19.870744    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:15:19.870886 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:19.870789    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz" event={"ID":"601ae44f-2916-47d8-9538-61dd910c49d7","Type":"ContainerStarted","Data":"44389ba1c42228e6cfcfe0f40475e4d4d71a5f473267e8efb5b22168467eac26"}
May 28 08:15:19.871045 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:19.871028    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:15:20.074535 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.074513    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:15:20.144300 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.144216    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-ca-trust-extracted\") pod \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") "
May 28 08:15:20.144300 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.144268    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-bound-sa-token\") pod \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") "
May 28 08:15:20.144300 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.144293    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-trusted-ca\") pod \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") "
May 28 08:15:20.144561 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.144316    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dfrz6\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-kube-api-access-dfrz6\") pod \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") "
May 28 08:15:20.144561 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.144342    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls\") pod \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") "
May 28 08:15:20.144561 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.144503    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-certificates\") pod \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") "
May 28 08:15:20.144561 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.144556    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-image-registry-private-configuration\") pod \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") "
May 28 08:15:20.144784 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.144604    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-installation-pull-secrets\") pod \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\" (UID: \"fa061e87-5cd2-4e3c-b6b0-eed405427b1c\") "
May 28 08:15:20.144835 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.144783    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-trusted-ca" (OuterVolumeSpecName: "trusted-ca") pod "fa061e87-5cd2-4e3c-b6b0-eed405427b1c" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c"). InnerVolumeSpecName "trusted-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:15:20.145400 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.144929    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca\" (UniqueName: \"kubernetes.io/configmap/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-trusted-ca\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:15:20.145400 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.145365    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-certificates" (OuterVolumeSpecName: "registry-certificates") pod "fa061e87-5cd2-4e3c-b6b0-eed405427b1c" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c"). InnerVolumeSpecName "registry-certificates". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:15:20.147001 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.146974    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-kube-api-access-dfrz6" (OuterVolumeSpecName: "kube-api-access-dfrz6") pod "fa061e87-5cd2-4e3c-b6b0-eed405427b1c" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c"). InnerVolumeSpecName "kube-api-access-dfrz6". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:15:20.147108 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.147047    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "fa061e87-5cd2-4e3c-b6b0-eed405427b1c" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:15:20.147164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.147127    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-image-registry-private-configuration" (OuterVolumeSpecName: "image-registry-private-configuration") pod "fa061e87-5cd2-4e3c-b6b0-eed405427b1c" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c"). InnerVolumeSpecName "image-registry-private-configuration". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:15:20.147940 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.147910    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-installation-pull-secrets" (OuterVolumeSpecName: "installation-pull-secrets") pod "fa061e87-5cd2-4e3c-b6b0-eed405427b1c" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c"). InnerVolumeSpecName "installation-pull-secrets". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:15:20.148069 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.148047    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-bound-sa-token" (OuterVolumeSpecName: "bound-sa-token") pod "fa061e87-5cd2-4e3c-b6b0-eed405427b1c" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c"). InnerVolumeSpecName "bound-sa-token". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:15:20.153393 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.153369    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-ca-trust-extracted" (OuterVolumeSpecName: "ca-trust-extracted") pod "fa061e87-5cd2-4e3c-b6b0-eed405427b1c" (UID: "fa061e87-5cd2-4e3c-b6b0-eed405427b1c"). InnerVolumeSpecName "ca-trust-extracted". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:15:20.221066 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.221037    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console-operator/console-operator-76dff48d59-xkwsz"
May 28 08:15:20.245556 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.245418    2577 reconciler_common.go:299] "Volume detached for volume \"ca-trust-extracted\" (UniqueName: \"kubernetes.io/empty-dir/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-ca-trust-extracted\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:15:20.245556 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.245451    2577 reconciler_common.go:299] "Volume detached for volume \"bound-sa-token\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-bound-sa-token\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:15:20.245556 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.245465    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dfrz6\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-kube-api-access-dfrz6\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:15:20.245556 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.245480    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/projected/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:15:20.245556 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.245494    2577 reconciler_common.go:299] "Volume detached for volume \"registry-certificates\" (UniqueName: \"kubernetes.io/configmap/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-registry-certificates\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:15:20.245556 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.245507    2577 reconciler_common.go:299] "Volume detached for volume \"image-registry-private-configuration\" (UniqueName: \"kubernetes.io/secret/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-image-registry-private-configuration\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:15:20.245556 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.245523    2577 reconciler_common.go:299] "Volume detached for volume \"installation-pull-secrets\" (UniqueName: \"kubernetes.io/secret/fa061e87-5cd2-4e3c-b6b0-eed405427b1c-installation-pull-secrets\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:15:20.386024 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.385986    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/downloads-57bd96b888-rgw82"]
May 28 08:15:20.386322 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.386309    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="fa061e87-5cd2-4e3c-b6b0-eed405427b1c" containerName="registry"
May 28 08:15:20.386379 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.386325    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="fa061e87-5cd2-4e3c-b6b0-eed405427b1c" containerName="registry"
May 28 08:15:20.386414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.386390    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="fa061e87-5cd2-4e3c-b6b0-eed405427b1c" containerName="registry"
May 28 08:15:20.390811 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.390790    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-rgw82"
May 28 08:15:20.393514 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.393491    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"default-dockercfg-dbx6b\""
May 28 08:15:20.393676 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.393533    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"openshift-service-ca.crt\""
May 28 08:15:20.393676 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.393574    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"kube-root-ca.crt\""
May 28 08:15:20.398984 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.398961    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-rgw82"]
May 28 08:15:20.446925 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.446893    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-j4b7l\" (UniqueName: \"kubernetes.io/projected/41681891-ca0e-4f41-9550-e312ceb051a7-kube-api-access-j4b7l\") pod \"downloads-57bd96b888-rgw82\" (UID: \"41681891-ca0e-4f41-9550-e312ceb051a7\") " pod="openshift-console/downloads-57bd96b888-rgw82"
May 28 08:15:20.547877 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.547834    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-j4b7l\" (UniqueName: \"kubernetes.io/projected/41681891-ca0e-4f41-9550-e312ceb051a7-kube-api-access-j4b7l\") pod \"downloads-57bd96b888-rgw82\" (UID: \"41681891-ca0e-4f41-9550-e312ceb051a7\") " pod="openshift-console/downloads-57bd96b888-rgw82"
May 28 08:15:20.556840 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.556812    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-j4b7l\" (UniqueName: \"kubernetes.io/projected/41681891-ca0e-4f41-9550-e312ceb051a7-kube-api-access-j4b7l\") pod \"downloads-57bd96b888-rgw82\" (UID: \"41681891-ca0e-4f41-9550-e312ceb051a7\") " pod="openshift-console/downloads-57bd96b888-rgw82"
May 28 08:15:20.700408 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.700322    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/downloads-57bd96b888-rgw82"
May 28 08:15:20.826712 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.826690    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/downloads-57bd96b888-rgw82"]
May 28 08:15:20.828708 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:15:20.828668    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod41681891_ca0e_4f41_9550_e312ceb051a7.slice/crio-52afa8246d20b9d6c6153f2a9b9ae020f712ce1966e4cac2850e7c5dd12bb423 WatchSource:0}: Error finding container 52afa8246d20b9d6c6153f2a9b9ae020f712ce1966e4cac2850e7c5dd12bb423: Status 404 returned error can't find the container with id 52afa8246d20b9d6c6153f2a9b9ae020f712ce1966e4cac2850e7c5dd12bb423
May 28 08:15:20.874931 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.874890    2577 generic.go:358] "Generic (PLEG): container finished" podID="fa061e87-5cd2-4e3c-b6b0-eed405427b1c" containerID="de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96" exitCode=0
May 28 08:15:20.875121 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.874961    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2"
May 28 08:15:20.875121 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.874979    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2" event={"ID":"fa061e87-5cd2-4e3c-b6b0-eed405427b1c","Type":"ContainerDied","Data":"de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96"}
May 28 08:15:20.875121 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.875015    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-image-registry/image-registry-798b45b98d-ctqd2" event={"ID":"fa061e87-5cd2-4e3c-b6b0-eed405427b1c","Type":"ContainerDied","Data":"21d41a438c481b0348e4ce4656d8c90ef39819a7d8f9813f71eedf1d4952cceb"}
May 28 08:15:20.875121 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.875031    2577 scope.go:117] "RemoveContainer" containerID="de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96"
May 28 08:15:20.876180 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.876154    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-rgw82" event={"ID":"41681891-ca0e-4f41-9550-e312ceb051a7","Type":"ContainerStarted","Data":"52afa8246d20b9d6c6153f2a9b9ae020f712ce1966e4cac2850e7c5dd12bb423"}
May 28 08:15:20.883045 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.883026    2577 scope.go:117] "RemoveContainer" containerID="de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96"
May 28 08:15:20.883307 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:15:20.883284    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96\": container with ID starting with de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96 not found: ID does not exist" containerID="de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96"
May 28 08:15:20.883393 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.883314    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96"} err="failed to get container status \"de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96\": rpc error: code = NotFound desc = could not find container \"de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96\": container with ID starting with de3da42f128e8592b1cf4d420602bb5ba2151ce11901e999e7a2676b6db4cc96 not found: ID does not exist"
May 28 08:15:20.892446 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.892424    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-image-registry/image-registry-798b45b98d-ctqd2"]
May 28 08:15:20.896072 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:20.896050    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-image-registry/image-registry-798b45b98d-ctqd2"]
May 28 08:15:22.242629 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:22.242596    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="fa061e87-5cd2-4e3c-b6b0-eed405427b1c" path="/var/lib/kubelet/pods/fa061e87-5cd2-4e3c-b6b0-eed405427b1c/volumes"
May 28 08:15:26.362879 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.362847    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-7cffb48f95-hbm56"]
May 28 08:15:26.367350 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.367325    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.371619 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.371570    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mftpg\""
May 28 08:15:26.371619 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.371587    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 28 08:15:26.371843 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.371623    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 28 08:15:26.371843 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.371645    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 28 08:15:26.371843 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.371588    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 28 08:15:26.371843 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.371588    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 28 08:15:26.377970 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.377946    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cffb48f95-hbm56"]
May 28 08:15:26.504936 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.504895    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-lhcqj\" (UniqueName: \"kubernetes.io/projected/5b4f1e68-0f78-432b-8846-00312dde7227-kube-api-access-lhcqj\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.505115 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.504961    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-service-ca\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.505115 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.504990    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b4f1e68-0f78-432b-8846-00312dde7227-console-serving-cert\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.505115 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.505030    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-oauth-serving-cert\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.505250 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.505135    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-console-config\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.505250 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.505168    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b4f1e68-0f78-432b-8846-00312dde7227-console-oauth-config\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.606225 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.606180    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-console-config\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.606418 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.606240    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b4f1e68-0f78-432b-8846-00312dde7227-console-oauth-config\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.606418 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.606276    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-lhcqj\" (UniqueName: \"kubernetes.io/projected/5b4f1e68-0f78-432b-8846-00312dde7227-kube-api-access-lhcqj\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.606418 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.606321    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-service-ca\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.606418 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.606354    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b4f1e68-0f78-432b-8846-00312dde7227-console-serving-cert\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.606418 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.606416    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-oauth-serving-cert\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.607019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.606995    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-console-config\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.607139 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.607083    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-oauth-serving-cert\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.607241 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.607215    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-service-ca\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.609516 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.609494    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b4f1e68-0f78-432b-8846-00312dde7227-console-oauth-config\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.609679 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.609637    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b4f1e68-0f78-432b-8846-00312dde7227-console-serving-cert\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.619565 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.619499    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-lhcqj\" (UniqueName: \"kubernetes.io/projected/5b4f1e68-0f78-432b-8846-00312dde7227-kube-api-access-lhcqj\") pod \"console-7cffb48f95-hbm56\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") " pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.678571 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.678530    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:26.824012 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.823975    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-7cffb48f95-hbm56"]
May 28 08:15:26.826709 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:15:26.826677    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod5b4f1e68_0f78_432b_8846_00312dde7227.slice/crio-6c810c16c5121441ec7e2e1b145de0f7aa2cbc922da72023e0b2253d89232ae7 WatchSource:0}: Error finding container 6c810c16c5121441ec7e2e1b145de0f7aa2cbc922da72023e0b2253d89232ae7: Status 404 returned error can't find the container with id 6c810c16c5121441ec7e2e1b145de0f7aa2cbc922da72023e0b2253d89232ae7
May 28 08:15:26.897834 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:26.897747    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cffb48f95-hbm56" event={"ID":"5b4f1e68-0f78-432b-8846-00312dde7227","Type":"ContainerStarted","Data":"6c810c16c5121441ec7e2e1b145de0f7aa2cbc922da72023e0b2253d89232ae7"}
May 28 08:15:29.910088 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:29.910045    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cffb48f95-hbm56" event={"ID":"5b4f1e68-0f78-432b-8846-00312dde7227","Type":"ContainerStarted","Data":"d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6"}
May 28 08:15:36.679429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:36.679377    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:36.679429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:36.679440    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:36.685181 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:36.685153    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:36.704534 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:36.704475    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-7cffb48f95-hbm56" podStartSLOduration=7.787352759 podStartE2EDuration="10.704455824s" podCreationTimestamp="2026-05-28 08:15:26 +0000 UTC" firstStartedPulling="2026-05-28 08:15:26.828782485 +0000 UTC m=+205.264961630" lastFinishedPulling="2026-05-28 08:15:29.74588554 +0000 UTC m=+208.182064695" observedRunningTime="2026-05-28 08:15:29.930004452 +0000 UTC m=+208.366183631" watchObservedRunningTime="2026-05-28 08:15:36.704455824 +0000 UTC m=+215.140634992"
May 28 08:15:36.935700 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:36.935603    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:15:37.935601 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:37.935511    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/downloads-57bd96b888-rgw82" event={"ID":"41681891-ca0e-4f41-9550-e312ceb051a7","Type":"ContainerStarted","Data":"e5e416489ba7cf73f7d32ac0e353e23030ed6072226c53206badb0242a6f9f17"}
May 28 08:15:37.936064 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:37.936008    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/downloads-57bd96b888-rgw82"
May 28 08:15:37.943859 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:37.943831    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/downloads-57bd96b888-rgw82"
May 28 08:15:37.954827 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:37.954767    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/downloads-57bd96b888-rgw82" podStartSLOduration=1.100352513 podStartE2EDuration="17.954749581s" podCreationTimestamp="2026-05-28 08:15:20 +0000 UTC" firstStartedPulling="2026-05-28 08:15:20.83067556 +0000 UTC m=+199.266854703" lastFinishedPulling="2026-05-28 08:15:37.685072626 +0000 UTC m=+216.121251771" observedRunningTime="2026-05-28 08:15:37.95384339 +0000 UTC m=+216.390022557" watchObservedRunningTime="2026-05-28 08:15:37.954749581 +0000 UTC m=+216.390928748"
May 28 08:15:45.278409 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:45.278370    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cffb48f95-hbm56"]
May 28 08:15:59.005833 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:59.005798    2577 generic.go:358] "Generic (PLEG): container finished" podID="ff8e5102-0c6f-4fd5-a002-7ad71a644f3b" containerID="2365dbfc95b2c3cb26d6772e5205e3118fec0eca6af75035d53aca164d26cb50" exitCode=0
May 28 08:15:59.006231 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:59.005875    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r" event={"ID":"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b","Type":"ContainerDied","Data":"2365dbfc95b2c3cb26d6772e5205e3118fec0eca6af75035d53aca164d26cb50"}
May 28 08:15:59.006231 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:15:59.006194    2577 scope.go:117] "RemoveContainer" containerID="2365dbfc95b2c3cb26d6772e5205e3118fec0eca6af75035d53aca164d26cb50"
May 28 08:16:00.010372 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:00.010328    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-service-ca-operator/service-ca-operator-57b5b68b8-zgn4r" event={"ID":"ff8e5102-0c6f-4fd5-a002-7ad71a644f3b","Type":"ContainerStarted","Data":"c0379e519c365f73460626f9ba7e47d1a68db4a184ecbcf873c620ba4747f149"}
May 28 08:16:02.017971 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:02.017935    2577 generic.go:358] "Generic (PLEG): container finished" podID="596d1656-db0e-41c6-8dc1-9926d1b470c9" containerID="5a0bba6828ba41106ca10f26a1d0fafdcbf34c9e48d215714a7e9201f7e1a9f1" exitCode=0
May 28 08:16:02.018349 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:02.018009    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr" event={"ID":"596d1656-db0e-41c6-8dc1-9926d1b470c9","Type":"ContainerDied","Data":"5a0bba6828ba41106ca10f26a1d0fafdcbf34c9e48d215714a7e9201f7e1a9f1"}
May 28 08:16:02.018349 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:02.018332    2577 scope.go:117] "RemoveContainer" containerID="5a0bba6828ba41106ca10f26a1d0fafdcbf34c9e48d215714a7e9201f7e1a9f1"
May 28 08:16:03.022404 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:03.022369    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-kube-storage-version-migrator-operator/kube-storage-version-migrator-operator-565644f49-qzjtr" event={"ID":"596d1656-db0e-41c6-8dc1-9926d1b470c9","Type":"ContainerStarted","Data":"f799959d92d6a06fd58f343764d72b5e706881d86e0f83938649e103e398fd03"}
May 28 08:16:10.302466 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.302403    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-7cffb48f95-hbm56" podUID="5b4f1e68-0f78-432b-8846-00312dde7227" containerName="console" containerID="cri-o://d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6" gracePeriod=15
May 28 08:16:10.567841 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.567818    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7cffb48f95-hbm56_5b4f1e68-0f78-432b-8846-00312dde7227/console/0.log"
May 28 08:16:10.567973 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.567881    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:16:10.713763 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.713730    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-console-config\") pod \"5b4f1e68-0f78-432b-8846-00312dde7227\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") "
May 28 08:16:10.713763 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.713770    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-service-ca\") pod \"5b4f1e68-0f78-432b-8846-00312dde7227\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") "
May 28 08:16:10.713968 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.713786    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-oauth-serving-cert\") pod \"5b4f1e68-0f78-432b-8846-00312dde7227\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") "
May 28 08:16:10.713968 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.713826    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b4f1e68-0f78-432b-8846-00312dde7227-console-oauth-config\") pod \"5b4f1e68-0f78-432b-8846-00312dde7227\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") "
May 28 08:16:10.713968 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.713845    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b4f1e68-0f78-432b-8846-00312dde7227-console-serving-cert\") pod \"5b4f1e68-0f78-432b-8846-00312dde7227\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") "
May 28 08:16:10.713968 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.713900    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-lhcqj\" (UniqueName: \"kubernetes.io/projected/5b4f1e68-0f78-432b-8846-00312dde7227-kube-api-access-lhcqj\") pod \"5b4f1e68-0f78-432b-8846-00312dde7227\" (UID: \"5b4f1e68-0f78-432b-8846-00312dde7227\") "
May 28 08:16:10.714223 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.714188    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-console-config" (OuterVolumeSpecName: "console-config") pod "5b4f1e68-0f78-432b-8846-00312dde7227" (UID: "5b4f1e68-0f78-432b-8846-00312dde7227"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:16:10.714223 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.714198    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-service-ca" (OuterVolumeSpecName: "service-ca") pod "5b4f1e68-0f78-432b-8846-00312dde7227" (UID: "5b4f1e68-0f78-432b-8846-00312dde7227"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:16:10.714375 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.714229    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "5b4f1e68-0f78-432b-8846-00312dde7227" (UID: "5b4f1e68-0f78-432b-8846-00312dde7227"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:16:10.716377 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.716351    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b4f1e68-0f78-432b-8846-00312dde7227-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "5b4f1e68-0f78-432b-8846-00312dde7227" (UID: "5b4f1e68-0f78-432b-8846-00312dde7227"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:16:10.716478 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.716350    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/5b4f1e68-0f78-432b-8846-00312dde7227-kube-api-access-lhcqj" (OuterVolumeSpecName: "kube-api-access-lhcqj") pod "5b4f1e68-0f78-432b-8846-00312dde7227" (UID: "5b4f1e68-0f78-432b-8846-00312dde7227"). InnerVolumeSpecName "kube-api-access-lhcqj". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:16:10.716478 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.716377    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/5b4f1e68-0f78-432b-8846-00312dde7227-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "5b4f1e68-0f78-432b-8846-00312dde7227" (UID: "5b4f1e68-0f78-432b-8846-00312dde7227"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:16:10.815280 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.815196    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-service-ca\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:10.815280 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.815227    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-oauth-serving-cert\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:10.815280 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.815242    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/5b4f1e68-0f78-432b-8846-00312dde7227-console-oauth-config\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:10.815280 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.815250    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/5b4f1e68-0f78-432b-8846-00312dde7227-console-serving-cert\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:10.815280 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.815259    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-lhcqj\" (UniqueName: \"kubernetes.io/projected/5b4f1e68-0f78-432b-8846-00312dde7227-kube-api-access-lhcqj\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:10.815280 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:10.815268    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/5b4f1e68-0f78-432b-8846-00312dde7227-console-config\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:11.047183 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:11.047158    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-7cffb48f95-hbm56_5b4f1e68-0f78-432b-8846-00312dde7227/console/0.log"
May 28 08:16:11.047337 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:11.047197    2577 generic.go:358] "Generic (PLEG): container finished" podID="5b4f1e68-0f78-432b-8846-00312dde7227" containerID="d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6" exitCode=2
May 28 08:16:11.047337 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:11.047255    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cffb48f95-hbm56" event={"ID":"5b4f1e68-0f78-432b-8846-00312dde7227","Type":"ContainerDied","Data":"d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6"}
May 28 08:16:11.047337 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:11.047263    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-7cffb48f95-hbm56"
May 28 08:16:11.047337 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:11.047284    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-7cffb48f95-hbm56" event={"ID":"5b4f1e68-0f78-432b-8846-00312dde7227","Type":"ContainerDied","Data":"6c810c16c5121441ec7e2e1b145de0f7aa2cbc922da72023e0b2253d89232ae7"}
May 28 08:16:11.047337 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:11.047300    2577 scope.go:117] "RemoveContainer" containerID="d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6"
May 28 08:16:11.056257 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:11.056240    2577 scope.go:117] "RemoveContainer" containerID="d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6"
May 28 08:16:11.056499 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:16:11.056480    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6\": container with ID starting with d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6 not found: ID does not exist" containerID="d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6"
May 28 08:16:11.056570 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:11.056506    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6"} err="failed to get container status \"d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6\": rpc error: code = NotFound desc = could not find container \"d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6\": container with ID starting with d954e9ecd7cff150a381ea5eaecc188eed7f16bf87a6f42bb54d6d9267a825c6 not found: ID does not exist"
May 28 08:16:11.069028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:11.068954    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-7cffb48f95-hbm56"]
May 28 08:16:11.073199 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:11.073174    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-7cffb48f95-hbm56"]
May 28 08:16:12.241919 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:12.241887    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="5b4f1e68-0f78-432b-8846-00312dde7227" path="/var/lib/kubelet/pods/5b4f1e68-0f78-432b-8846-00312dde7227/volumes"
May 28 08:16:12.935487 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:12.935447    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:16:12.938055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:12.938018    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-certs\" (UniqueName: \"kubernetes.io/secret/90e9cadb-328a-47e1-a119-bae6987ac229-metrics-certs\") pod \"network-metrics-daemon-mlpsc\" (UID: \"90e9cadb-328a-47e1-a119-bae6987ac229\") " pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:16:13.141848 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:13.141817    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-multus\"/\"metrics-daemon-sa-dockercfg-s6dkv\""
May 28 08:16:13.149453 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:13.149427    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-multus/network-metrics-daemon-mlpsc"
May 28 08:16:13.276439 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:13.276410    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-multus/network-metrics-daemon-mlpsc"]
May 28 08:16:13.278968 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:16:13.278927    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod90e9cadb_328a_47e1_a119_bae6987ac229.slice/crio-83b535fb163d7e4dde89acc77a54a33f14eddd5b58a29e6ea6dd9d4179e9d1e9 WatchSource:0}: Error finding container 83b535fb163d7e4dde89acc77a54a33f14eddd5b58a29e6ea6dd9d4179e9d1e9: Status 404 returned error can't find the container with id 83b535fb163d7e4dde89acc77a54a33f14eddd5b58a29e6ea6dd9d4179e9d1e9
May 28 08:16:14.057790 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:14.057750    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mlpsc" event={"ID":"90e9cadb-328a-47e1-a119-bae6987ac229","Type":"ContainerStarted","Data":"83b535fb163d7e4dde89acc77a54a33f14eddd5b58a29e6ea6dd9d4179e9d1e9"}
May 28 08:16:15.063268 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:15.063230    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mlpsc" event={"ID":"90e9cadb-328a-47e1-a119-bae6987ac229","Type":"ContainerStarted","Data":"28f6e4d0a647289d8950447d12c593415e315a13b1f4889bbdc026220180e390"}
May 28 08:16:15.063268 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:15.063270    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-multus/network-metrics-daemon-mlpsc" event={"ID":"90e9cadb-328a-47e1-a119-bae6987ac229","Type":"ContainerStarted","Data":"5d7be3b7227c9593e1daf00705698b3bf2d001a6840d63ebbb74f8f3f86195e1"}
May 28 08:16:15.081241 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:15.081190    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-multus/network-metrics-daemon-mlpsc" podStartSLOduration=251.62210675 podStartE2EDuration="4m13.081172867s" podCreationTimestamp="2026-05-28 08:12:02 +0000 UTC" firstStartedPulling="2026-05-28 08:16:13.280797517 +0000 UTC m=+251.716976662" lastFinishedPulling="2026-05-28 08:16:14.739863633 +0000 UTC m=+253.176042779" observedRunningTime="2026-05-28 08:16:15.080744802 +0000 UTC m=+253.516923969" watchObservedRunningTime="2026-05-28 08:16:15.081172867 +0000 UTC m=+253.517352067"
May 28 08:16:26.146873 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.146792    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 08:16:26.147363 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.147200    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="alertmanager" containerID="cri-o://24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc" gracePeriod=120
May 28 08:16:26.147363 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.147247    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="kube-rbac-proxy-metric" containerID="cri-o://832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1" gracePeriod=120
May 28 08:16:26.147363 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.147270    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="prom-label-proxy" containerID="cri-o://24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc" gracePeriod=120
May 28 08:16:26.147363 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.147282    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="kube-rbac-proxy-web" containerID="cri-o://4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f" gracePeriod=120
May 28 08:16:26.147363 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.147334    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="config-reloader" containerID="cri-o://a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a" gracePeriod=120
May 28 08:16:26.147363 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.147282    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-monitoring/alertmanager-main-0" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="kube-rbac-proxy" containerID="cri-o://6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3" gracePeriod=120
May 28 08:16:26.353356 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.353321    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-console/console-5856dc4d7d-h6d84"]
May 28 08:16:26.353623 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.353610    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="5b4f1e68-0f78-432b-8846-00312dde7227" containerName="console"
May 28 08:16:26.353688 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.353624    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="5b4f1e68-0f78-432b-8846-00312dde7227" containerName="console"
May 28 08:16:26.353732 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.353721    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="5b4f1e68-0f78-432b-8846-00312dde7227" containerName="console"
May 28 08:16:26.355534 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.355516    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.358837 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.358818    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-serving-cert\""
May 28 08:16:26.359633 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.359612    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-oauth-config\""
May 28 08:16:26.359750 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.359615    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"oauth-serving-cert\""
May 28 08:16:26.360060 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.360043    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"console-config\""
May 28 08:16:26.361144 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.361121    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-console\"/\"console-dockercfg-mftpg\""
May 28 08:16:26.361254 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.361131    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"service-ca\""
May 28 08:16:26.367036 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.367018    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-console\"/\"trusted-ca-bundle\""
May 28 08:16:26.371405 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.371383    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5856dc4d7d-h6d84"]
May 28 08:16:26.450371 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.450281    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-trusted-ca-bundle\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.450371 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.450340    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-oauth-serving-cert\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.450544 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.450387    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-config\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.450544 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.450404    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-97t74\" (UniqueName: \"kubernetes.io/projected/4a60cfd4-1703-423d-8b96-03c1e1c64cab-kube-api-access-97t74\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.450544 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.450424    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-serving-cert\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.450544 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.450496    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-oauth-config\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.450693 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.450547    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-service-ca\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.551737 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.551689    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-service-ca\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.551737 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.551745    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-trusted-ca-bundle\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.551999 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.551806    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-oauth-serving-cert\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.551999 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.551835    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-config\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.551999 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.551854    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-97t74\" (UniqueName: \"kubernetes.io/projected/4a60cfd4-1703-423d-8b96-03c1e1c64cab-kube-api-access-97t74\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.551999 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.551889    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-serving-cert\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.552199 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.552014    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-oauth-config\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.552521 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.552469    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-service-ca\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.552521 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.552503    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-oauth-serving-cert\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.552907 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.552542    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-config\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.552974 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.552962    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-trusted-ca-bundle\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.554418 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.554396    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-oauth-config\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.554612 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.554590    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-serving-cert\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.560823 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.560800    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-97t74\" (UniqueName: \"kubernetes.io/projected/4a60cfd4-1703-423d-8b96-03c1e1c64cab-kube-api-access-97t74\") pod \"console-5856dc4d7d-h6d84\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") " pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.665476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.665433    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:26.792249 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:26.792194    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-console/console-5856dc4d7d-h6d84"]
May 28 08:16:26.794338 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:16:26.794312    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod4a60cfd4_1703_423d_8b96_03c1e1c64cab.slice/crio-e168aa0696d3e66c7f5caf6bcebedc264db8c71f6ff97ee945102a7229c3b9aa WatchSource:0}: Error finding container e168aa0696d3e66c7f5caf6bcebedc264db8c71f6ff97ee945102a7229c3b9aa: Status 404 returned error can't find the container with id e168aa0696d3e66c7f5caf6bcebedc264db8c71f6ff97ee945102a7229c3b9aa
May 28 08:16:27.104249 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.104214    2577 generic.go:358] "Generic (PLEG): container finished" podID="faf2d625-97c7-48b9-826b-567b708ee418" containerID="24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc" exitCode=0
May 28 08:16:27.104249 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.104242    2577 generic.go:358] "Generic (PLEG): container finished" podID="faf2d625-97c7-48b9-826b-567b708ee418" containerID="6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3" exitCode=0
May 28 08:16:27.104249 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.104253    2577 generic.go:358] "Generic (PLEG): container finished" podID="faf2d625-97c7-48b9-826b-567b708ee418" containerID="a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a" exitCode=0
May 28 08:16:27.104486 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.104261    2577 generic.go:358] "Generic (PLEG): container finished" podID="faf2d625-97c7-48b9-826b-567b708ee418" containerID="24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc" exitCode=0
May 28 08:16:27.104486 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.104252    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerDied","Data":"24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc"}
May 28 08:16:27.104486 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.104339    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerDied","Data":"6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3"}
May 28 08:16:27.104486 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.104357    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerDied","Data":"a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a"}
May 28 08:16:27.104486 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.104368    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerDied","Data":"24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc"}
May 28 08:16:27.105613 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.105591    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5856dc4d7d-h6d84" event={"ID":"4a60cfd4-1703-423d-8b96-03c1e1c64cab","Type":"ContainerStarted","Data":"38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac"}
May 28 08:16:27.105752 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.105615    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5856dc4d7d-h6d84" event={"ID":"4a60cfd4-1703-423d-8b96-03c1e1c64cab","Type":"ContainerStarted","Data":"e168aa0696d3e66c7f5caf6bcebedc264db8c71f6ff97ee945102a7229c3b9aa"}
May 28 08:16:27.124398 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.124340    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-console/console-5856dc4d7d-h6d84" podStartSLOduration=1.124327433 podStartE2EDuration="1.124327433s" podCreationTimestamp="2026-05-28 08:16:26 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:16:27.122169798 +0000 UTC m=+265.558348976" watchObservedRunningTime="2026-05-28 08:16:27.124327433 +0000 UTC m=+265.560506598"
May 28 08:16:27.395027 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.395002    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:27.562670 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.562614    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/faf2d625-97c7-48b9-826b-567b708ee418-metrics-client-ca\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.562871 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.562726    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy-metric\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.562871 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.562751    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy-web\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.562871 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.562776    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-web-config\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.562871 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.562808    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/faf2d625-97c7-48b9-826b-567b708ee418-config-out\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.562871 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.562851    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-cluster-tls-config\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.563145 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.562886    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/faf2d625-97c7-48b9-826b-567b708ee418-alertmanager-main-db\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.563145 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.562919    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/faf2d625-97c7-48b9-826b-567b708ee418-tls-assets\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.563145 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.562973    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-main-tls\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.563145 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.562999    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-config-volume\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.563145 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.563029    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8nnn9\" (UniqueName: \"kubernetes.io/projected/faf2d625-97c7-48b9-826b-567b708ee418-kube-api-access-8nnn9\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.563145 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.563031    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faf2d625-97c7-48b9-826b-567b708ee418-metrics-client-ca" (OuterVolumeSpecName: "metrics-client-ca") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "metrics-client-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:16:27.563145 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.563068    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.563145 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.563098    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faf2d625-97c7-48b9-826b-567b708ee418-alertmanager-trusted-ca-bundle\") pod \"faf2d625-97c7-48b9-826b-567b708ee418\" (UID: \"faf2d625-97c7-48b9-826b-567b708ee418\") "
May 28 08:16:27.563709 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.563317    2577 reconciler_common.go:299] "Volume detached for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/faf2d625-97c7-48b9-826b-567b708ee418-metrics-client-ca\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:27.563935 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.563907    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/faf2d625-97c7-48b9-826b-567b708ee418-alertmanager-trusted-ca-bundle" (OuterVolumeSpecName: "alertmanager-trusted-ca-bundle") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "alertmanager-trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:16:27.565735 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.565709    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-main-tls" (OuterVolumeSpecName: "secret-alertmanager-main-tls") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "secret-alertmanager-main-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:16:27.565968 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.565948    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faf2d625-97c7-48b9-826b-567b708ee418-alertmanager-main-db" (OuterVolumeSpecName: "alertmanager-main-db") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "alertmanager-main-db". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:16:27.566038 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.565980    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/faf2d625-97c7-48b9-826b-567b708ee418-config-out" (OuterVolumeSpecName: "config-out") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "config-out". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:16:27.566381 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.566298    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faf2d625-97c7-48b9-826b-567b708ee418-kube-api-access-8nnn9" (OuterVolumeSpecName: "kube-api-access-8nnn9") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "kube-api-access-8nnn9". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:16:27.566475 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.566414    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy-metric" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-metric") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-metric". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:16:27.566608 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.566588    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy-web" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy-web") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy-web". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:16:27.566697 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.566607    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy" (OuterVolumeSpecName: "secret-alertmanager-kube-rbac-proxy") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "secret-alertmanager-kube-rbac-proxy". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:16:27.568254 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.568236    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/faf2d625-97c7-48b9-826b-567b708ee418-tls-assets" (OuterVolumeSpecName: "tls-assets") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "tls-assets". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:16:27.568254 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.568236    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-config-volume" (OuterVolumeSpecName: "config-volume") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "config-volume". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:16:27.571337 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.571315    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-cluster-tls-config" (OuterVolumeSpecName: "cluster-tls-config") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "cluster-tls-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:16:27.577821 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.577799    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-web-config" (OuterVolumeSpecName: "web-config") pod "faf2d625-97c7-48b9-826b-567b708ee418" (UID: "faf2d625-97c7-48b9-826b-567b708ee418"). InnerVolumeSpecName "web-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:16:27.663747 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.663638    2577 reconciler_common.go:299] "Volume detached for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/faf2d625-97c7-48b9-826b-567b708ee418-tls-assets\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:27.663747 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.663699    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-main-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:27.663747 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.663711    2577 reconciler_common.go:299] "Volume detached for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-config-volume\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:27.663747 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.663721    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8nnn9\" (UniqueName: \"kubernetes.io/projected/faf2d625-97c7-48b9-826b-567b708ee418-kube-api-access-8nnn9\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:27.663747 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.663730    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:27.663747 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.663740    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/faf2d625-97c7-48b9-826b-567b708ee418-alertmanager-trusted-ca-bundle\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:27.663747 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.663750    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy-metric\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:27.663747 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.663759    2577 reconciler_common.go:299] "Volume detached for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-secret-alertmanager-kube-rbac-proxy-web\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:27.664082 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.663770    2577 reconciler_common.go:299] "Volume detached for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-web-config\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:27.664082 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.663779    2577 reconciler_common.go:299] "Volume detached for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/faf2d625-97c7-48b9-826b-567b708ee418-config-out\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:27.664082 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.663786    2577 reconciler_common.go:299] "Volume detached for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/faf2d625-97c7-48b9-826b-567b708ee418-cluster-tls-config\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:27.664082 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:27.663794    2577 reconciler_common.go:299] "Volume detached for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/faf2d625-97c7-48b9-826b-567b708ee418-alertmanager-main-db\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:16:28.111856 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.111816    2577 generic.go:358] "Generic (PLEG): container finished" podID="faf2d625-97c7-48b9-826b-567b708ee418" containerID="832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1" exitCode=0
May 28 08:16:28.111856 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.111844    2577 generic.go:358] "Generic (PLEG): container finished" podID="faf2d625-97c7-48b9-826b-567b708ee418" containerID="4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f" exitCode=0
May 28 08:16:28.112082 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.111898    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerDied","Data":"832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1"}
May 28 08:16:28.112082 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.111916    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.112082 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.111938    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerDied","Data":"4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f"}
May 28 08:16:28.112082 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.111953    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"faf2d625-97c7-48b9-826b-567b708ee418","Type":"ContainerDied","Data":"ab1371f7ad3f3ef09e798fc5a335af53841787b5191370a3c6bbb9c6150032f5"}
May 28 08:16:28.112082 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.111981    2577 scope.go:117] "RemoveContainer" containerID="24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc"
May 28 08:16:28.125306 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.125284    2577 scope.go:117] "RemoveContainer" containerID="832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1"
May 28 08:16:28.132001 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.131982    2577 scope.go:117] "RemoveContainer" containerID="6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3"
May 28 08:16:28.138649 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.138625    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 08:16:28.138839 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.138816    2577 scope.go:117] "RemoveContainer" containerID="4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f"
May 28 08:16:28.143264 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.143240    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 08:16:28.146324 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.146308    2577 scope.go:117] "RemoveContainer" containerID="a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a"
May 28 08:16:28.153240 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.153223    2577 scope.go:117] "RemoveContainer" containerID="24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc"
May 28 08:16:28.168603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.168577    2577 scope.go:117] "RemoveContainer" containerID="01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485"
May 28 08:16:28.171429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171406    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 08:16:28.171769 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171752    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="init-config-reloader"
May 28 08:16:28.171855 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171772    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="init-config-reloader"
May 28 08:16:28.171855 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171786    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="kube-rbac-proxy"
May 28 08:16:28.171855 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171794    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="kube-rbac-proxy"
May 28 08:16:28.171855 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171813    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="prom-label-proxy"
May 28 08:16:28.171855 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171821    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="prom-label-proxy"
May 28 08:16:28.171855 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171837    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="alertmanager"
May 28 08:16:28.171855 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171845    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="alertmanager"
May 28 08:16:28.171855 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171852    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="kube-rbac-proxy-web"
May 28 08:16:28.172279 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171861    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="kube-rbac-proxy-web"
May 28 08:16:28.172279 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171881    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="config-reloader"
May 28 08:16:28.172279 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171890    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="config-reloader"
May 28 08:16:28.172279 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171899    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="kube-rbac-proxy-metric"
May 28 08:16:28.172279 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171906    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="kube-rbac-proxy-metric"
May 28 08:16:28.172279 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171978    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="prom-label-proxy"
May 28 08:16:28.172279 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.171993    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="kube-rbac-proxy"
May 28 08:16:28.172279 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.172008    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="alertmanager"
May 28 08:16:28.172279 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.172016    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="config-reloader"
May 28 08:16:28.172279 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.172026    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="kube-rbac-proxy-metric"
May 28 08:16:28.172279 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.172036    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="faf2d625-97c7-48b9-826b-567b708ee418" containerName="kube-rbac-proxy-web"
May 28 08:16:28.175513 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.175494    2577 scope.go:117] "RemoveContainer" containerID="24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc"
May 28 08:16:28.175792 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:16:28.175776    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc\": container with ID starting with 24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc not found: ID does not exist" containerID="24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc"
May 28 08:16:28.175834 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.175803    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc"} err="failed to get container status \"24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc\": rpc error: code = NotFound desc = could not find container \"24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc\": container with ID starting with 24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc not found: ID does not exist"
May 28 08:16:28.175834 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.175820    2577 scope.go:117] "RemoveContainer" containerID="832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1"
May 28 08:16:28.176028 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:16:28.176014    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1\": container with ID starting with 832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1 not found: ID does not exist" containerID="832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1"
May 28 08:16:28.176076 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.176031    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1"} err="failed to get container status \"832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1\": rpc error: code = NotFound desc = could not find container \"832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1\": container with ID starting with 832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1 not found: ID does not exist"
May 28 08:16:28.176076 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.176042    2577 scope.go:117] "RemoveContainer" containerID="6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3"
May 28 08:16:28.176235 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:16:28.176220    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3\": container with ID starting with 6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3 not found: ID does not exist" containerID="6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3"
May 28 08:16:28.176289 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.176238    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3"} err="failed to get container status \"6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3\": rpc error: code = NotFound desc = could not find container \"6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3\": container with ID starting with 6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3 not found: ID does not exist"
May 28 08:16:28.176289 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.176250    2577 scope.go:117] "RemoveContainer" containerID="4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f"
May 28 08:16:28.176430 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:16:28.176409    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f\": container with ID starting with 4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f not found: ID does not exist" containerID="4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f"
May 28 08:16:28.176479 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.176433    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f"} err="failed to get container status \"4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f\": rpc error: code = NotFound desc = could not find container \"4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f\": container with ID starting with 4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f not found: ID does not exist"
May 28 08:16:28.176479 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.176445    2577 scope.go:117] "RemoveContainer" containerID="a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a"
May 28 08:16:28.176577 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.176501    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.176627 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:16:28.176608    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a\": container with ID starting with a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a not found: ID does not exist" containerID="a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a"
May 28 08:16:28.176692 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.176629    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a"} err="failed to get container status \"a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a\": rpc error: code = NotFound desc = could not find container \"a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a\": container with ID starting with a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a not found: ID does not exist"
May 28 08:16:28.176692 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.176646    2577 scope.go:117] "RemoveContainer" containerID="24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc"
May 28 08:16:28.177192 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:16:28.177169    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc\": container with ID starting with 24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc not found: ID does not exist" containerID="24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc"
May 28 08:16:28.177292 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.177197    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc"} err="failed to get container status \"24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc\": rpc error: code = NotFound desc = could not find container \"24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc\": container with ID starting with 24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc not found: ID does not exist"
May 28 08:16:28.177292 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.177211    2577 scope.go:117] "RemoveContainer" containerID="01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485"
May 28 08:16:28.177495 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:16:28.177427    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485\": container with ID starting with 01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485 not found: ID does not exist" containerID="01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485"
May 28 08:16:28.177495 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.177448    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485"} err="failed to get container status \"01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485\": rpc error: code = NotFound desc = could not find container \"01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485\": container with ID starting with 01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485 not found: ID does not exist"
May 28 08:16:28.177495 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.177463    2577 scope.go:117] "RemoveContainer" containerID="24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc"
May 28 08:16:28.177720 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.177701    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc"} err="failed to get container status \"24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc\": rpc error: code = NotFound desc = could not find container \"24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc\": container with ID starting with 24078a508f32aca1b65d9ce70b2726fbfeca45f88451ca1530e342a22edfc2fc not found: ID does not exist"
May 28 08:16:28.177777 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.177721    2577 scope.go:117] "RemoveContainer" containerID="832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1"
May 28 08:16:28.177963 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.177943    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1"} err="failed to get container status \"832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1\": rpc error: code = NotFound desc = could not find container \"832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1\": container with ID starting with 832826640f2a49f7fc5aeca3e09a79e2d647f3bcae6f6999cf9f3c36337937c1 not found: ID does not exist"
May 28 08:16:28.178005 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.177964    2577 scope.go:117] "RemoveContainer" containerID="6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3"
May 28 08:16:28.178221 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.178203    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3"} err="failed to get container status \"6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3\": rpc error: code = NotFound desc = could not find container \"6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3\": container with ID starting with 6bad803ffa0f2e0e25a4f77f6bdb14bb0c7e9ceb765b818d7b4e34ea0e0275f3 not found: ID does not exist"
May 28 08:16:28.178267 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.178223    2577 scope.go:117] "RemoveContainer" containerID="4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f"
May 28 08:16:28.178443 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.178427    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f"} err="failed to get container status \"4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f\": rpc error: code = NotFound desc = could not find container \"4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f\": container with ID starting with 4b01e91b7faa47a3a987eab6ac242bde4d161259b88fc62c3e2cd23b8739d55f not found: ID does not exist"
May 28 08:16:28.178488 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.178444    2577 scope.go:117] "RemoveContainer" containerID="a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a"
May 28 08:16:28.178707 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.178683    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a"} err="failed to get container status \"a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a\": rpc error: code = NotFound desc = could not find container \"a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a\": container with ID starting with a676e0240cedef07b63ac4362369a0b382cf28f2cd3caae079353530d895136a not found: ID does not exist"
May 28 08:16:28.178707 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.178708    2577 scope.go:117] "RemoveContainer" containerID="24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc"
May 28 08:16:28.178934 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.178919    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc"} err="failed to get container status \"24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc\": rpc error: code = NotFound desc = could not find container \"24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc\": container with ID starting with 24b2ea7796f61cd01774783b6f31bc4313468add32f31475ed6561dee3d24ecc not found: ID does not exist"
May 28 08:16:28.178984 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.178935    2577 scope.go:117] "RemoveContainer" containerID="01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485"
May 28 08:16:28.179185 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.179160    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485"} err="failed to get container status \"01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485\": rpc error: code = NotFound desc = could not find container \"01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485\": container with ID starting with 01289ce9df894e8b68ce58647921ab5838bf9994f1c4e7fea8c106aa2a5dc485 not found: ID does not exist"
May 28 08:16:28.179413 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.179399    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy\""
May 28 08:16:28.179480 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.179401    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-cluster-tls-config\""
May 28 08:16:28.179549 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.179529    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-generated\""
May 28 08:16:28.179620 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.179566    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls-assets-0\""
May 28 08:16:28.180006 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.179989    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-web-config\""
May 28 08:16:28.180676 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.180559    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-dockercfg-hxcjg\""
May 28 08:16:28.180676 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.180573    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-web\""
May 28 08:16:28.180676 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.180559    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-kube-rbac-proxy-metric\""
May 28 08:16:28.180676 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.180576    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-monitoring\"/\"alertmanager-main-tls\""
May 28 08:16:28.184829 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.184812    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-monitoring\"/\"alertmanager-trusted-ca-bundle\""
May 28 08:16:28.188764 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.188743    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 08:16:28.241741 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.241708    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="faf2d625-97c7-48b9-826b-567b708ee418" path="/var/lib/kubelet/pods/faf2d625-97c7-48b9-826b-567b708ee418/volumes"
May 28 08:16:28.269289 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269252    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.269418 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269297    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.269418 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269325    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.269487 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269451    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-config-volume\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.269530 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269502    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.269581 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269568    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-web-config\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.269635 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269595    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.269635 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269625    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.269776 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269725    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-jxjbk\" (UniqueName: \"kubernetes.io/projected/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-kube-api-access-jxjbk\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.269835 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269780    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.269887 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269840    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.269939 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269901    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.269989 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.269933    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-config-out\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.370702 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.370584    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.370702 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.370638    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-web-config\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.370888 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.370761    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.370888 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.370794    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.370888 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.370836    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-jxjbk\" (UniqueName: \"kubernetes.io/projected/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-kube-api-access-jxjbk\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.370888 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.370869    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.371078 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.371009    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.371078 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.371055    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.371179 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.371081    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-config-out\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.371179 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.371123    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.371179 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.371153    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.371342 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.371183    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.371342 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.371219    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-config-volume\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.371342 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.371319    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-main-db\" (UniqueName: \"kubernetes.io/empty-dir/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-alertmanager-main-db\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.371616 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.371519    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"metrics-client-ca\" (UniqueName: \"kubernetes.io/configmap/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-metrics-client-ca\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.372756 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.372462    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"alertmanager-trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-alertmanager-trusted-ca-bundle\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.374091 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.374062    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-metric\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-secret-alertmanager-kube-rbac-proxy-metric\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.374353 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.374331    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"web-config\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-web-config\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.374458 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.374384    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"cluster-tls-config\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-cluster-tls-config\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.374883 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.374862    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy-web\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-secret-alertmanager-kube-rbac-proxy-web\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.374969 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.374946    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-volume\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-config-volume\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.375200 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.375182    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"config-out\" (UniqueName: \"kubernetes.io/empty-dir/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-config-out\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.375245 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.375191    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-main-tls\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-secret-alertmanager-main-tls\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.375958 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.375942    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"secret-alertmanager-kube-rbac-proxy\" (UniqueName: \"kubernetes.io/secret/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-secret-alertmanager-kube-rbac-proxy\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.376041 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.376023    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"tls-assets\" (UniqueName: \"kubernetes.io/projected/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-tls-assets\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.379983 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.379961    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-jxjbk\" (UniqueName: \"kubernetes.io/projected/e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa-kube-api-access-jxjbk\") pod \"alertmanager-main-0\" (UID: \"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa\") " pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.487296 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.487259    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-monitoring/alertmanager-main-0"
May 28 08:16:28.622194 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:28.622108    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-monitoring/alertmanager-main-0"]
May 28 08:16:28.624307 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:16:28.624277    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pode3dc6bb1_b7a0_49b5_ac8b_77ec50bd9baa.slice/crio-c7aaccce6a45fa3e5afff05fdd74bfaeabff9302b17e2fdccea924d38bbe7bb7 WatchSource:0}: Error finding container c7aaccce6a45fa3e5afff05fdd74bfaeabff9302b17e2fdccea924d38bbe7bb7: Status 404 returned error can't find the container with id c7aaccce6a45fa3e5afff05fdd74bfaeabff9302b17e2fdccea924d38bbe7bb7
May 28 08:16:29.116937 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:29.116905    2577 generic.go:358] "Generic (PLEG): container finished" podID="e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa" containerID="592624b17695c7b39d8527faa64852d50f857497a0c81e871012851ab9e76d76" exitCode=0
May 28 08:16:29.117148 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:29.116978    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa","Type":"ContainerDied","Data":"592624b17695c7b39d8527faa64852d50f857497a0c81e871012851ab9e76d76"}
May 28 08:16:29.117148 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:29.117007    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa","Type":"ContainerStarted","Data":"c7aaccce6a45fa3e5afff05fdd74bfaeabff9302b17e2fdccea924d38bbe7bb7"}
May 28 08:16:30.130145 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:30.130105    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa","Type":"ContainerStarted","Data":"66af5a6077a9b44c5ca64084494f8c741e0cd712a9f09c90c79dabdbc760e97c"}
May 28 08:16:30.130145 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:30.130147    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa","Type":"ContainerStarted","Data":"b87571815521e85a368f7a43aa6fd652121cab660e4d0c026f1428c3548e32f6"}
May 28 08:16:30.130702 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:30.130159    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa","Type":"ContainerStarted","Data":"be5af05e66265b2a9b00c3c21a68b9eb8c43305e3e785b7fec3f8bf45685fd17"}
May 28 08:16:30.130702 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:30.130171    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa","Type":"ContainerStarted","Data":"59dbfc9e540952296f7027a16628bc02082ce7ed3cc2e21b58e530bf1ad1ac1e"}
May 28 08:16:30.130702 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:30.130181    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa","Type":"ContainerStarted","Data":"b28ac09d857402eb1e0bb7dd6db1cceb3c8b9cc23ffe82f2ca4c4812484335cf"}
May 28 08:16:30.130702 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:30.130191    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-monitoring/alertmanager-main-0" event={"ID":"e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa","Type":"ContainerStarted","Data":"c063f761c7da1c0d97d7ae2d46d466b3dd7e92dce1205fe64de19f412fe19594"}
May 28 08:16:30.159918 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:30.159852    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-monitoring/alertmanager-main-0" podStartSLOduration=2.159831873 podStartE2EDuration="2.159831873s" podCreationTimestamp="2026-05-28 08:16:28 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:16:30.157686362 +0000 UTC m=+268.593865530" watchObservedRunningTime="2026-05-28 08:16:30.159831873 +0000 UTC m=+268.596011053"
May 28 08:16:36.665870 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:36.665823    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:36.665870 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:36.665871    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:36.671054 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:36.671032    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:37.156485 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:37.156452    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:16:49.696819 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.696783    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["kube-system/global-pull-secret-syncer-nwdqc"]
May 28 08:16:49.699959 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.699937    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwdqc"
May 28 08:16:49.702765 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.702742    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"kube-system\"/\"original-pull-secret\""
May 28 08:16:49.707159 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.707129    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nwdqc"]
May 28 08:16:49.747054 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.747021    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/18b9e331-35ec-44e5-9025-158761237d3d-original-pull-secret\") pod \"global-pull-secret-syncer-nwdqc\" (UID: \"18b9e331-35ec-44e5-9025-158761237d3d\") " pod="kube-system/global-pull-secret-syncer-nwdqc"
May 28 08:16:49.747226 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.747064    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/18b9e331-35ec-44e5-9025-158761237d3d-kubelet-config\") pod \"global-pull-secret-syncer-nwdqc\" (UID: \"18b9e331-35ec-44e5-9025-158761237d3d\") " pod="kube-system/global-pull-secret-syncer-nwdqc"
May 28 08:16:49.747226 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.747169    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/18b9e331-35ec-44e5-9025-158761237d3d-dbus\") pod \"global-pull-secret-syncer-nwdqc\" (UID: \"18b9e331-35ec-44e5-9025-158761237d3d\") " pod="kube-system/global-pull-secret-syncer-nwdqc"
May 28 08:16:49.848039 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.848009    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/18b9e331-35ec-44e5-9025-158761237d3d-original-pull-secret\") pod \"global-pull-secret-syncer-nwdqc\" (UID: \"18b9e331-35ec-44e5-9025-158761237d3d\") " pod="kube-system/global-pull-secret-syncer-nwdqc"
May 28 08:16:49.848182 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.848050    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/18b9e331-35ec-44e5-9025-158761237d3d-kubelet-config\") pod \"global-pull-secret-syncer-nwdqc\" (UID: \"18b9e331-35ec-44e5-9025-158761237d3d\") " pod="kube-system/global-pull-secret-syncer-nwdqc"
May 28 08:16:49.848182 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.848075    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/18b9e331-35ec-44e5-9025-158761237d3d-dbus\") pod \"global-pull-secret-syncer-nwdqc\" (UID: \"18b9e331-35ec-44e5-9025-158761237d3d\") " pod="kube-system/global-pull-secret-syncer-nwdqc"
May 28 08:16:49.848182 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.848148    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kubelet-config\" (UniqueName: \"kubernetes.io/host-path/18b9e331-35ec-44e5-9025-158761237d3d-kubelet-config\") pod \"global-pull-secret-syncer-nwdqc\" (UID: \"18b9e331-35ec-44e5-9025-158761237d3d\") " pod="kube-system/global-pull-secret-syncer-nwdqc"
May 28 08:16:49.848284 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.848226    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"dbus\" (UniqueName: \"kubernetes.io/host-path/18b9e331-35ec-44e5-9025-158761237d3d-dbus\") pod \"global-pull-secret-syncer-nwdqc\" (UID: \"18b9e331-35ec-44e5-9025-158761237d3d\") " pod="kube-system/global-pull-secret-syncer-nwdqc"
May 28 08:16:49.850442 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:49.850413    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"original-pull-secret\" (UniqueName: \"kubernetes.io/secret/18b9e331-35ec-44e5-9025-158761237d3d-original-pull-secret\") pod \"global-pull-secret-syncer-nwdqc\" (UID: \"18b9e331-35ec-44e5-9025-158761237d3d\") " pod="kube-system/global-pull-secret-syncer-nwdqc"
May 28 08:16:50.010207 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:50.010164    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="kube-system/global-pull-secret-syncer-nwdqc"
May 28 08:16:50.136534 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:50.136497    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["kube-system/global-pull-secret-syncer-nwdqc"]
May 28 08:16:50.138921 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:16:50.138876    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod18b9e331_35ec_44e5_9025_158761237d3d.slice/crio-c13e3594bbad0662224f71bf434b644f26000a4b558fbf11f8598e0e8c57646e WatchSource:0}: Error finding container c13e3594bbad0662224f71bf434b644f26000a4b558fbf11f8598e0e8c57646e: Status 404 returned error can't find the container with id c13e3594bbad0662224f71bf434b644f26000a4b558fbf11f8598e0e8c57646e
May 28 08:16:50.190786 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:50.190746    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nwdqc" event={"ID":"18b9e331-35ec-44e5-9025-158761237d3d","Type":"ContainerStarted","Data":"c13e3594bbad0662224f71bf434b644f26000a4b558fbf11f8598e0e8c57646e"}
May 28 08:16:55.206777 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:55.206741    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="kube-system/global-pull-secret-syncer-nwdqc" event={"ID":"18b9e331-35ec-44e5-9025-158761237d3d","Type":"ContainerStarted","Data":"f1291a640ab8ef30c64a320f2fd612c7d69df276f6b0945f5b28f96ba51ac307"}
May 28 08:16:55.222775 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:16:55.222729    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="kube-system/global-pull-secret-syncer-nwdqc" podStartSLOduration=1.8408169079999999 podStartE2EDuration="6.222715655s" podCreationTimestamp="2026-05-28 08:16:49 +0000 UTC" firstStartedPulling="2026-05-28 08:16:50.140907115 +0000 UTC m=+288.577086258" lastFinishedPulling="2026-05-28 08:16:54.522805852 +0000 UTC m=+292.958985005" observedRunningTime="2026-05-28 08:16:55.221640118 +0000 UTC m=+293.657819285" watchObservedRunningTime="2026-05-28 08:16:55.222715655 +0000 UTC m=+293.658894823"
May 28 08:17:02.104777 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:17:02.104750    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:17:02.105217 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:17:02.105134    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:17:02.121207 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:17:02.121186    2577 kubelet.go:1628] "Image garbage collection succeeded"
May 28 08:19:08.537721 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.537679    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"]
May 28 08:19:08.541366 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.541346    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.544157 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.544122    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 08:19:08.544274 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.544136    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 08:19:08.544415 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.544397    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 08:19:08.544492 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.544414    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-intra-comm\""
May 28 08:19:08.544492 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.544400    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 08:19:08.544598 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.544489    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 08:19:08.544598 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.544568    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-hdq7q\""
May 28 08:19:08.544726 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.544611    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 08:19:08.551803 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.551777    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"]
May 28 08:19:08.713818 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.713775    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-offline-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.714019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.713900    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-feast-data\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.714019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.713936    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-online-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.714019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.713967    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-registry-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.714019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.713995    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-6sgbh\" (UniqueName: \"kubernetes.io/projected/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-kube-api-access-6sgbh\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.714188 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.714059    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-ui-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.814854 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.814758    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-offline-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.815035 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.814858    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-feast-data\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.815035 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.814878    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-online-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.815035 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.814907    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-registry-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.815035 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:19:08.814922    2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 08:19:08.815035 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.814929    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-6sgbh\" (UniqueName: \"kubernetes.io/projected/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-kube-api-access-6sgbh\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.815035 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:19:08.815026    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-offline-tls podName:6849a93b-a29a-441c-b107-9cc7e2ceeeeb nodeName:}" failed. No retries permitted until 2026-05-28 08:19:09.31500495 +0000 UTC m=+427.751184095 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-offline-tls") pod "feast-simple-feast-setup-5b6596f8cb-xrgj5" (UID: "6849a93b-a29a-441c-b107-9cc7e2ceeeeb") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 08:19:08.815417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.815076    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-ui-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.815417 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.815378    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-feast-data\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.817592 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.817563    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-registry-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.817708 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.817575    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-ui-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.817781 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.817766    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-online-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:08.824756 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:08.824718    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-6sgbh\" (UniqueName: \"kubernetes.io/projected/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-kube-api-access-6sgbh\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:09.319129 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:09.319083    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-offline-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:09.321585 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:09.321564    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-offline-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-xrgj5\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:09.454675 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:09.454631    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:09.589046 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:09.589008    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"]
May 28 08:19:09.592701 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:19:09.592650    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod6849a93b_a29a_441c_b107_9cc7e2ceeeeb.slice/crio-298e167eb15d4873149eca0a115774a8bbbcc3ebae77932a9907644ab106950b WatchSource:0}: Error finding container 298e167eb15d4873149eca0a115774a8bbbcc3ebae77932a9907644ab106950b: Status 404 returned error can't find the container with id 298e167eb15d4873149eca0a115774a8bbbcc3ebae77932a9907644ab106950b
May 28 08:19:09.595044 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:09.595026    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 08:19:10.599176 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:10.599137    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerStarted","Data":"298e167eb15d4873149eca0a115774a8bbbcc3ebae77932a9907644ab106950b"}
May 28 08:19:43.705113 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:43.704155    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerStarted","Data":"a8e3fb8bd2f958473bb1be0df46e01383958853a37eb75dab9dc641f3afcc812"}
May 28 08:19:48.719939 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:48.719901    2577 generic.go:358] "Generic (PLEG): container finished" podID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerID="a8e3fb8bd2f958473bb1be0df46e01383958853a37eb75dab9dc641f3afcc812" exitCode=0
May 28 08:19:48.720452 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:48.719976    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerDied","Data":"a8e3fb8bd2f958473bb1be0df46e01383958853a37eb75dab9dc641f3afcc812"}
May 28 08:19:49.724250 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:49.724215    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerStarted","Data":"ea231e4e2a706d01a734f9ff702a0992a233d6fd4689cb8b164db8d1ba0bc5b4"}
May 28 08:19:54.739626 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:54.739542    2577 generic.go:358] "Generic (PLEG): container finished" podID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerID="ea231e4e2a706d01a734f9ff702a0992a233d6fd4689cb8b164db8d1ba0bc5b4" exitCode=0
May 28 08:19:54.739626 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:54.739575    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerDied","Data":"ea231e4e2a706d01a734f9ff702a0992a233d6fd4689cb8b164db8d1ba0bc5b4"}
May 28 08:19:55.747043 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:55.746999    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerStarted","Data":"5edbbfb185b825522b0f5e7bd6cbc4fe59088dec7c17764c9c82a56b42d5a397"}
May 28 08:19:55.747549 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:55.747058    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerStarted","Data":"bec2007e3f7589d69a9af74e1194cd86af8c34503edec5770ed90b8aeb304d94"}
May 28 08:19:55.747549 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:55.747074    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerStarted","Data":"4c273fb5172fad72f4bf7719c526c490470913eee8d6cd5d4095589f73387288"}
May 28 08:19:55.747549 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:55.747089    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerStarted","Data":"349ef6b5a704197f875195627fb1c1d157e26d6c501daa4c16d3b2cb7dfe0ae4"}
May 28 08:19:55.790239 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:55.789919    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podStartSLOduration=13.985912463 podStartE2EDuration="47.789895778s" podCreationTimestamp="2026-05-28 08:19:08 +0000 UTC" firstStartedPulling="2026-05-28 08:19:09.595155732 +0000 UTC m=+428.031334875" lastFinishedPulling="2026-05-28 08:19:43.399139039 +0000 UTC m=+461.835318190" observedRunningTime="2026-05-28 08:19:55.788346743 +0000 UTC m=+474.224525909" watchObservedRunningTime="2026-05-28 08:19:55.789895778 +0000 UTC m=+474.226074945"
May 28 08:19:57.455385 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:57.455333    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:57.455888 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:57.455402    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:57.455888 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:57.455417    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:57.455888 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:57.455429    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:57.457820 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:57.457780    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 28 08:19:57.457820 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:57.457807    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.24:6571: connect: connection refused"
May 28 08:19:57.458040 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:57.457807    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.24:8816: connect: connection refused"
May 28 08:19:57.458040 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:57.457868    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 28 08:19:59.455737 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:59.455550    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:59.455737 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:59.455613    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:59.455737 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:59.455628    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:19:59.455737 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:19:59.455641    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:20:00.456640 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:20:00.456610    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:20:00.457118 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:20:00.456762    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:20:00.457217 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:20:00.457193    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:20:00.457466 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:20:00.457439    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:20:00.457578 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:20:00.457509    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:20:00.457747 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:20:00.457732    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:20:00.461426 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:20:00.461409    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:20:00.769005 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:20:00.768972    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:20:40.026850 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:20:40.026808    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5856dc4d7d-h6d84"]
May 28 08:21:01.835494 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:01.835409    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"]
May 28 08:21:01.836048 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:01.835765    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="registry" containerID="cri-o://349ef6b5a704197f875195627fb1c1d157e26d6c501daa4c16d3b2cb7dfe0ae4" gracePeriod=30
May 28 08:21:01.836048 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:01.835837    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="online" containerID="cri-o://4c273fb5172fad72f4bf7719c526c490470913eee8d6cd5d4095589f73387288" gracePeriod=30
May 28 08:21:01.836048 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:01.835836    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="offline" containerID="cri-o://bec2007e3f7589d69a9af74e1194cd86af8c34503edec5770ed90b8aeb304d94" gracePeriod=30
May 28 08:21:01.836048 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:01.835943    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="ui" containerID="cri-o://5edbbfb185b825522b0f5e7bd6cbc4fe59088dec7c17764c9c82a56b42d5a397" gracePeriod=30
May 28 08:21:02.089938 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.089831    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"]
May 28 08:21:02.094734 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.094705    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.097707 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.097678    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-ms98q\""
May 28 08:21:02.109552 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.106287    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"]
May 28 08:21:02.112242 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.112180    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-w9pqb\" (UniqueName: \"kubernetes.io/projected/8d915380-4306-4285-8803-95bbec28d0a7-kube-api-access-w9pqb\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.112559 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.112480    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-registry-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.112760 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.112721    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-offline-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.112999 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.112891    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-ui-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.112999 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.112948    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d915380-4306-4285-8803-95bbec28d0a7-feast-data\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.113270 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.112976    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-online-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.214194 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.214152    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-w9pqb\" (UniqueName: \"kubernetes.io/projected/8d915380-4306-4285-8803-95bbec28d0a7-kube-api-access-w9pqb\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.214688 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.214572    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-registry-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.215091 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.215068    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-offline-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.215237 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.215222    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-ui-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.215758 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.215641    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d915380-4306-4285-8803-95bbec28d0a7-feast-data\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.215908 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.215893    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-online-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.216193 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:21:02.214789    2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-registry-tls: secret "feast-simple-feast-setup-registry-tls" not found
May 28 08:21:02.216561 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:21:02.216548    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-registry-tls podName:8d915380-4306-4285-8803-95bbec28d0a7 nodeName:}" failed. No retries permitted until 2026-05-28 08:21:02.716428433 +0000 UTC m=+541.152607593 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-registry-tls") pod "feast-simple-feast-setup-5b6596f8cb-4p4xx" (UID: "8d915380-4306-4285-8803-95bbec28d0a7") : secret "feast-simple-feast-setup-registry-tls" not found
May 28 08:21:02.217327 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.217306    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d915380-4306-4285-8803-95bbec28d0a7-feast-data\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.217508 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:21:02.215539    2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 08:21:02.217705 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:21:02.217694    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-ui-tls podName:8d915380-4306-4285-8803-95bbec28d0a7 nodeName:}" failed. No retries permitted until 2026-05-28 08:21:02.71764844 +0000 UTC m=+541.153827594 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-ui-tls") pod "feast-simple-feast-setup-5b6596f8cb-4p4xx" (UID: "8d915380-4306-4285-8803-95bbec28d0a7") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 08:21:02.217833 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:21:02.215587    2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-offline-tls: secret "feast-simple-feast-setup-offline-tls" not found
May 28 08:21:02.217972 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:21:02.217962    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-offline-tls podName:8d915380-4306-4285-8803-95bbec28d0a7 nodeName:}" failed. No retries permitted until 2026-05-28 08:21:02.717946724 +0000 UTC m=+541.154125882 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-offline-tls") pod "feast-simple-feast-setup-5b6596f8cb-4p4xx" (UID: "8d915380-4306-4285-8803-95bbec28d0a7") : secret "feast-simple-feast-setup-offline-tls" not found
May 28 08:21:02.218085 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:21:02.216180    2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 28 08:21:02.218225 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:21:02.218215    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-online-tls podName:8d915380-4306-4285-8803-95bbec28d0a7 nodeName:}" failed. No retries permitted until 2026-05-28 08:21:02.718201894 +0000 UTC m=+541.154381057 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-online-tls") pod "feast-simple-feast-setup-5b6596f8cb-4p4xx" (UID: "8d915380-4306-4285-8803-95bbec28d0a7") : secret "feast-simple-feast-setup-online-tls" not found
May 28 08:21:02.225428 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.225401    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-w9pqb\" (UniqueName: \"kubernetes.io/projected/8d915380-4306-4285-8803-95bbec28d0a7-kube-api-access-w9pqb\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.720938 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.720889    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-registry-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.721141 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.720982    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-offline-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.721141 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.721011    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-ui-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.721141 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.721040    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-online-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.723701 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.723646    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-online-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.723819 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.723788    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-offline-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.723859 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.723843    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-registry-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.723893 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.723880    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-ui-tls\") pod \"feast-simple-feast-setup-5b6596f8cb-4p4xx\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") " pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:02.959414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.959318    2577 generic.go:358] "Generic (PLEG): container finished" podID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerID="5edbbfb185b825522b0f5e7bd6cbc4fe59088dec7c17764c9c82a56b42d5a397" exitCode=0
May 28 08:21:02.959414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.959348    2577 generic.go:358] "Generic (PLEG): container finished" podID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerID="4c273fb5172fad72f4bf7719c526c490470913eee8d6cd5d4095589f73387288" exitCode=0
May 28 08:21:02.959414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.959380    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerDied","Data":"5edbbfb185b825522b0f5e7bd6cbc4fe59088dec7c17764c9c82a56b42d5a397"}
May 28 08:21:02.959414 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:02.959402    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerDied","Data":"4c273fb5172fad72f4bf7719c526c490470913eee8d6cd5d4095589f73387288"}
May 28 08:21:03.013860 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:03.013814    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:03.158131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:03.158101    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"]
May 28 08:21:03.160351 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:21:03.160327    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod8d915380_4306_4285_8803_95bbec28d0a7.slice/crio-f58e6f509851ac42e270eae3644f9c61e0a09eb54f823f3a6152e73b1db80011 WatchSource:0}: Error finding container f58e6f509851ac42e270eae3644f9c61e0a09eb54f823f3a6152e73b1db80011: Status 404 returned error can't find the container with id f58e6f509851ac42e270eae3644f9c61e0a09eb54f823f3a6152e73b1db80011
May 28 08:21:03.964254 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:03.964213    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerStarted","Data":"3ac972b90a9ea4ad167ecd3bd0c42a2ac0dac2a6a67129def44de178d9935c98"}
May 28 08:21:03.964254 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:03.964250    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerStarted","Data":"f58e6f509851ac42e270eae3644f9c61e0a09eb54f823f3a6152e73b1db80011"}
May 28 08:21:05.049822 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.049776    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-console/console-5856dc4d7d-h6d84" podUID="4a60cfd4-1703-423d-8b96-03c1e1c64cab" containerName="console" containerID="cri-o://38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac" gracePeriod=15
May 28 08:21:05.298310 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.298282    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5856dc4d7d-h6d84_4a60cfd4-1703-423d-8b96-03c1e1c64cab/console/0.log"
May 28 08:21:05.298421 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.298367    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:21:05.342362 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.342285    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-97t74\" (UniqueName: \"kubernetes.io/projected/4a60cfd4-1703-423d-8b96-03c1e1c64cab-kube-api-access-97t74\") pod \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") "
May 28 08:21:05.342362 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.342323    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-trusted-ca-bundle\") pod \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") "
May 28 08:21:05.342362 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.342341    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-oauth-serving-cert\") pod \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") "
May 28 08:21:05.342362 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.342363    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-oauth-config\") pod \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") "
May 28 08:21:05.342672 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.342423    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-config\") pod \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") "
May 28 08:21:05.342672 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.342462    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-serving-cert\") pod \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") "
May 28 08:21:05.342672 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.342500    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-service-ca\") pod \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\" (UID: \"4a60cfd4-1703-423d-8b96-03c1e1c64cab\") "
May 28 08:21:05.342810 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.342728    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-oauth-serving-cert" (OuterVolumeSpecName: "oauth-serving-cert") pod "4a60cfd4-1703-423d-8b96-03c1e1c64cab" (UID: "4a60cfd4-1703-423d-8b96-03c1e1c64cab"). InnerVolumeSpecName "oauth-serving-cert". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:21:05.342871 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.342822    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-trusted-ca-bundle" (OuterVolumeSpecName: "trusted-ca-bundle") pod "4a60cfd4-1703-423d-8b96-03c1e1c64cab" (UID: "4a60cfd4-1703-423d-8b96-03c1e1c64cab"). InnerVolumeSpecName "trusted-ca-bundle". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:21:05.342871 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.342837    2577 reconciler_common.go:299] "Volume detached for volume \"oauth-serving-cert\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-oauth-serving-cert\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:05.343155 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.343122    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-service-ca" (OuterVolumeSpecName: "service-ca") pod "4a60cfd4-1703-423d-8b96-03c1e1c64cab" (UID: "4a60cfd4-1703-423d-8b96-03c1e1c64cab"). InnerVolumeSpecName "service-ca". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:21:05.343317 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.343298    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-config" (OuterVolumeSpecName: "console-config") pod "4a60cfd4-1703-423d-8b96-03c1e1c64cab" (UID: "4a60cfd4-1703-423d-8b96-03c1e1c64cab"). InnerVolumeSpecName "console-config". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:21:05.344834 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.344808    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/4a60cfd4-1703-423d-8b96-03c1e1c64cab-kube-api-access-97t74" (OuterVolumeSpecName: "kube-api-access-97t74") pod "4a60cfd4-1703-423d-8b96-03c1e1c64cab" (UID: "4a60cfd4-1703-423d-8b96-03c1e1c64cab"). InnerVolumeSpecName "kube-api-access-97t74". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:21:05.344951 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.344875    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-serving-cert" (OuterVolumeSpecName: "console-serving-cert") pod "4a60cfd4-1703-423d-8b96-03c1e1c64cab" (UID: "4a60cfd4-1703-423d-8b96-03c1e1c64cab"). InnerVolumeSpecName "console-serving-cert". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:21:05.345178 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.345162    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-oauth-config" (OuterVolumeSpecName: "console-oauth-config") pod "4a60cfd4-1703-423d-8b96-03c1e1c64cab" (UID: "4a60cfd4-1703-423d-8b96-03c1e1c64cab"). InnerVolumeSpecName "console-oauth-config". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:21:05.444006 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.443958    2577 reconciler_common.go:299] "Volume detached for volume \"console-config\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-config\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:05.444006 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.443990    2577 reconciler_common.go:299] "Volume detached for volume \"console-serving-cert\" (UniqueName: \"kubernetes.io/secret/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-serving-cert\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:05.444006 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.444000    2577 reconciler_common.go:299] "Volume detached for volume \"service-ca\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-service-ca\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:05.444006 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.444012    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-97t74\" (UniqueName: \"kubernetes.io/projected/4a60cfd4-1703-423d-8b96-03c1e1c64cab-kube-api-access-97t74\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:05.444006 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.444022    2577 reconciler_common.go:299] "Volume detached for volume \"trusted-ca-bundle\" (UniqueName: \"kubernetes.io/configmap/4a60cfd4-1703-423d-8b96-03c1e1c64cab-trusted-ca-bundle\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:05.444418 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.444030    2577 reconciler_common.go:299] "Volume detached for volume \"console-oauth-config\" (UniqueName: \"kubernetes.io/secret/4a60cfd4-1703-423d-8b96-03c1e1c64cab-console-oauth-config\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:05.972940 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.972910    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_console-5856dc4d7d-h6d84_4a60cfd4-1703-423d-8b96-03c1e1c64cab/console/0.log"
May 28 08:21:05.973143 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.972949    2577 generic.go:358] "Generic (PLEG): container finished" podID="4a60cfd4-1703-423d-8b96-03c1e1c64cab" containerID="38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac" exitCode=2
May 28 08:21:05.973143 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.972992    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5856dc4d7d-h6d84" event={"ID":"4a60cfd4-1703-423d-8b96-03c1e1c64cab","Type":"ContainerDied","Data":"38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac"}
May 28 08:21:05.973143 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.973015    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-console/console-5856dc4d7d-h6d84" event={"ID":"4a60cfd4-1703-423d-8b96-03c1e1c64cab","Type":"ContainerDied","Data":"e168aa0696d3e66c7f5caf6bcebedc264db8c71f6ff97ee945102a7229c3b9aa"}
May 28 08:21:05.973143 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.973024    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-console/console-5856dc4d7d-h6d84"
May 28 08:21:05.973143 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.973031    2577 scope.go:117] "RemoveContainer" containerID="38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac"
May 28 08:21:05.981566 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.981546    2577 scope.go:117] "RemoveContainer" containerID="38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac"
May 28 08:21:05.981945 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:21:05.981925    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac\": container with ID starting with 38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac not found: ID does not exist" containerID="38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac"
May 28 08:21:05.982035 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.981951    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac"} err="failed to get container status \"38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac\": rpc error: code = NotFound desc = could not find container \"38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac\": container with ID starting with 38a2780817343f323f475adac90a6cd88f8c80c0c1dccf9b81a45489817bd1ac not found: ID does not exist"
May 28 08:21:05.994391 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.994362    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-console/console-5856dc4d7d-h6d84"]
May 28 08:21:05.998512 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:05.998485    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-console/console-5856dc4d7d-h6d84"]
May 28 08:21:06.242631 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:06.242551    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="4a60cfd4-1703-423d-8b96-03c1e1c64cab" path="/var/lib/kubelet/pods/4a60cfd4-1703-423d-8b96-03c1e1c64cab/volumes"
May 28 08:21:07.982107 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:07.982070    2577 generic.go:358] "Generic (PLEG): container finished" podID="8d915380-4306-4285-8803-95bbec28d0a7" containerID="3ac972b90a9ea4ad167ecd3bd0c42a2ac0dac2a6a67129def44de178d9935c98" exitCode=0
May 28 08:21:07.982504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:07.982147    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerDied","Data":"3ac972b90a9ea4ad167ecd3bd0c42a2ac0dac2a6a67129def44de178d9935c98"}
May 28 08:21:08.986735 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:08.986700    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerStarted","Data":"dd59ca99cc0431bfe20cacca1afeb429b92edf60f4b82ef8187b02e6d4be2c9b"}
May 28 08:21:10.458146 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:10.458095    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 28 08:21:10.766776 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:10.766732    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 28 08:21:14.002928 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:14.002891    2577 generic.go:358] "Generic (PLEG): container finished" podID="8d915380-4306-4285-8803-95bbec28d0a7" containerID="dd59ca99cc0431bfe20cacca1afeb429b92edf60f4b82ef8187b02e6d4be2c9b" exitCode=0
May 28 08:21:14.003418 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:14.002969    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerDied","Data":"dd59ca99cc0431bfe20cacca1afeb429b92edf60f4b82ef8187b02e6d4be2c9b"}
May 28 08:21:15.010370 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.010319    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerStarted","Data":"e3cb7eb70438104149af8e6549eb13898db09fc43a3bb44c718c7300a7491c64"}
May 28 08:21:15.010370 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.010367    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerStarted","Data":"7d49d166a6c02504239613fcd7f8d80f7e24b3e15783cc5b24795af3df584475"}
May 28 08:21:15.010951 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.010381    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerStarted","Data":"92d023dd6c55e222c80e6b6f897c116260ae69de2007741b1732216ca7e608dd"}
May 28 08:21:15.010951 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.010394    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerStarted","Data":"4b2951fd0458c662ddc19a46a055704fe7b83eebd1d7f2e630af1dd28d795ef6"}
May 28 08:21:15.014731 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.014636    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:15.014731 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.014694    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:15.014731 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.014708    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:15.014731 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.014720    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:15.017019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.016982    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 28 08:21:15.017151 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.017013    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.25:6571: connect: connection refused"
May 28 08:21:15.017151 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.017100    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 28 08:21:15.017309 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.017286    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused"
May 28 08:21:15.062134 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:15.062059    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podStartSLOduration=13.062039099 podStartE2EDuration="13.062039099s" podCreationTimestamp="2026-05-28 08:21:02 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:21:15.060538038 +0000 UTC m=+553.496717232" watchObservedRunningTime="2026-05-28 08:21:15.062039099 +0000 UTC m=+553.498218265"
May 28 08:21:18.015294 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:18.015248    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.25:6571: connect: connection refused"
May 28 08:21:18.015814 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:18.015248    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.25:8816: connect: connection refused"
May 28 08:21:18.015814 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:18.015271    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 28 08:21:18.015814 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:18.015336    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 28 08:21:20.458320 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:20.458269    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 28 08:21:20.766741 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:20.766698    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 28 08:21:21.015632 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:21.015600    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:21.015809 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:21.015720    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:21.015951 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:21.015931    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:21.016291 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:21.016262    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:21.016404 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:21.016300    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:21.016404 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:21.016315    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:21.016530 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:21.016465    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:21.016589 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:21.016571    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:21.016737 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:21.016723    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:21.020094 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:21.020055    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:21.033359 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:21.033339    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:21.036248 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:21.036228    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:21:30.457552 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:30.457500    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.24:8443: connect: connection refused"
May 28 08:21:30.458021 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:30.457643    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:21:30.766504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:30.766465    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="online" probeResult="failure" output="Get \"https://10.134.0.24:6567/health\": dial tcp 10.134.0.24:6567: connect: connection refused"
May 28 08:21:30.766684 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:30.766579    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:21:32.066757 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.066721    2577 generic.go:358] "Generic (PLEG): container finished" podID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerID="bec2007e3f7589d69a9af74e1194cd86af8c34503edec5770ed90b8aeb304d94" exitCode=137
May 28 08:21:32.066757 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.066750    2577 generic.go:358] "Generic (PLEG): container finished" podID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerID="349ef6b5a704197f875195627fb1c1d157e26d6c501daa4c16d3b2cb7dfe0ae4" exitCode=137
May 28 08:21:32.067168 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.066792    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerDied","Data":"bec2007e3f7589d69a9af74e1194cd86af8c34503edec5770ed90b8aeb304d94"}
May 28 08:21:32.067168 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.066823    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerDied","Data":"349ef6b5a704197f875195627fb1c1d157e26d6c501daa4c16d3b2cb7dfe0ae4"}
May 28 08:21:32.491814 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.491790    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:21:32.587598 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.587555    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-online-tls\") pod \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") "
May 28 08:21:32.587598 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.587593    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-6sgbh\" (UniqueName: \"kubernetes.io/projected/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-kube-api-access-6sgbh\") pod \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") "
May 28 08:21:32.587894 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.587617    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-ui-tls\") pod \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") "
May 28 08:21:32.587894 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.587705    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-offline-tls\") pod \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") "
May 28 08:21:32.587894 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.587738    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-feast-data\") pod \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") "
May 28 08:21:32.587894 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.587802    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-registry-tls\") pod \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\" (UID: \"6849a93b-a29a-441c-b107-9cc7e2ceeeeb\") "
May 28 08:21:32.588518 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.588360    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-feast-data" (OuterVolumeSpecName: "feast-data") pod "6849a93b-a29a-441c-b107-9cc7e2ceeeeb" (UID: "6849a93b-a29a-441c-b107-9cc7e2ceeeeb"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:21:32.590174 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.590138    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "6849a93b-a29a-441c-b107-9cc7e2ceeeeb" (UID: "6849a93b-a29a-441c-b107-9cc7e2ceeeeb"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:21:32.590270 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.590226    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-kube-api-access-6sgbh" (OuterVolumeSpecName: "kube-api-access-6sgbh") pod "6849a93b-a29a-441c-b107-9cc7e2ceeeeb" (UID: "6849a93b-a29a-441c-b107-9cc7e2ceeeeb"). InnerVolumeSpecName "kube-api-access-6sgbh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:21:32.590270 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.590256    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "6849a93b-a29a-441c-b107-9cc7e2ceeeeb" (UID: "6849a93b-a29a-441c-b107-9cc7e2ceeeeb"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:21:32.590343 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.590284    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "6849a93b-a29a-441c-b107-9cc7e2ceeeeb" (UID: "6849a93b-a29a-441c-b107-9cc7e2ceeeeb"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:21:32.590343 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.590303    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-online-tls" (OuterVolumeSpecName: "online-tls") pod "6849a93b-a29a-441c-b107-9cc7e2ceeeeb" (UID: "6849a93b-a29a-441c-b107-9cc7e2ceeeeb"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:21:32.689166 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.689078    2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-online-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:32.689166 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.689110    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-6sgbh\" (UniqueName: \"kubernetes.io/projected/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-kube-api-access-6sgbh\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:32.689166 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.689121    2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-ui-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:32.689166 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.689132    2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-offline-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:32.689166 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.689141    2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-feast-data\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:32.689166 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:32.689148    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/6849a93b-a29a-441c-b107-9cc7e2ceeeeb-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:21:33.073122 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:33.073085    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5" event={"ID":"6849a93b-a29a-441c-b107-9cc7e2ceeeeb","Type":"ContainerDied","Data":"298e167eb15d4873149eca0a115774a8bbbcc3ebae77932a9907644ab106950b"}
May 28 08:21:33.073122 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:33.073118    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"
May 28 08:21:33.073572 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:33.073139    2577 scope.go:117] "RemoveContainer" containerID="5edbbfb185b825522b0f5e7bd6cbc4fe59088dec7c17764c9c82a56b42d5a397"
May 28 08:21:33.082531 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:33.082514    2577 scope.go:117] "RemoveContainer" containerID="bec2007e3f7589d69a9af74e1194cd86af8c34503edec5770ed90b8aeb304d94"
May 28 08:21:33.090277 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:33.090255    2577 scope.go:117] "RemoveContainer" containerID="4c273fb5172fad72f4bf7719c526c490470913eee8d6cd5d4095589f73387288"
May 28 08:21:33.096383 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:33.096358    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"]
May 28 08:21:33.098470 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:33.098450    2577 scope.go:117] "RemoveContainer" containerID="349ef6b5a704197f875195627fb1c1d157e26d6c501daa4c16d3b2cb7dfe0ae4"
May 28 08:21:33.100234 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:33.100209    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5b6596f8cb-xrgj5"]
May 28 08:21:33.106065 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:33.106043    2577 scope.go:117] "RemoveContainer" containerID="ea231e4e2a706d01a734f9ff702a0992a233d6fd4689cb8b164db8d1ba0bc5b4"
May 28 08:21:33.113272 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:33.113250    2577 scope.go:117] "RemoveContainer" containerID="a8e3fb8bd2f958473bb1be0df46e01383958853a37eb75dab9dc641f3afcc812"
May 28 08:21:34.241907 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:21:34.241871    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" path="/var/lib/kubelet/pods/6849a93b-a29a-441c-b107-9cc7e2ceeeeb/volumes"
May 28 08:22:02.134899 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:02.134874    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:22:02.136085 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:02.136067    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:22:23.841148 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841052    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"]
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841480    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="online"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841494    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="online"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841517    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="4a60cfd4-1703-423d-8b96-03c1e1c64cab" containerName="console"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841526    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="4a60cfd4-1703-423d-8b96-03c1e1c64cab" containerName="console"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841538    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="feast-apply"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841546    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="feast-apply"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841554    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="offline"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841564    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="offline"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841581    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="registry"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841589    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="registry"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841597    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="ui"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841605    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="ui"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841636    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="feast-init"
May 28 08:22:23.841716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841644    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="feast-init"
May 28 08:22:23.842349 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841728    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="ui"
May 28 08:22:23.842349 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841745    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="4a60cfd4-1703-423d-8b96-03c1e1c64cab" containerName="console"
May 28 08:22:23.842349 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841756    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="offline"
May 28 08:22:23.842349 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841767    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="online"
May 28 08:22:23.842349 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.841777    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="6849a93b-a29a-441c-b107-9cc7e2ceeeeb" containerName="registry"
May 28 08:22:23.848982 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.848947    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:23.852891 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.852867    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 08:22:23.853621 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.853594    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 08:22:23.853787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.853683    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 08:22:23.853906 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.853888    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 08:22:23.854149 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.854106    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-intra-comm\""
May 28 08:22:23.854309 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.854176    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-pzfx4\""
May 28 08:22:23.854390 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.854227    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 08:22:23.854916 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.854896    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 08:22:23.861005 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.860972    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"]
May 28 08:22:23.923450 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.923406    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-rd2fz\" (UniqueName: \"kubernetes.io/projected/22332100-a3b6-45c7-acd0-3a5c690c4ff3-kube-api-access-rd2fz\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:23.923450 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.923457    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/22332100-a3b6-45c7-acd0-3a5c690c4ff3-registry-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:23.923694 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.923529    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-online-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:23.923694 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.923563    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-ui-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:23.923694 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.923595    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-offline-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:23.923694 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:23.923616    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22332100-a3b6-45c7-acd0-3a5c690c4ff3-feast-data\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.024981 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.024940    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/22332100-a3b6-45c7-acd0-3a5c690c4ff3-registry-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.025172 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.025027    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-online-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.025172 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.025078    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-ui-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.025172 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.025119    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-offline-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.025172 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.025153    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22332100-a3b6-45c7-acd0-3a5c690c4ff3-feast-data\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.025373 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.025185    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-rd2fz\" (UniqueName: \"kubernetes.io/projected/22332100-a3b6-45c7-acd0-3a5c690c4ff3-kube-api-access-rd2fz\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.025373 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:22:24.025198    2577 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found
May 28 08:22:24.025373 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:22:24.025276    2577 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 08:22:24.025373 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:22:24.025286    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-ui-tls podName:22332100-a3b6-45c7-acd0-3a5c690c4ff3 nodeName:}" failed. No retries permitted until 2026-05-28 08:22:24.525260733 +0000 UTC m=+622.961439888 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-ui-tls") pod "feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" (UID: "22332100-a3b6-45c7-acd0-3a5c690c4ff3") : secret "feast-simple-feast-remote-setup-ui-tls" not found
May 28 08:22:24.025594 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:22:24.025376    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-offline-tls podName:22332100-a3b6-45c7-acd0-3a5c690c4ff3 nodeName:}" failed. No retries permitted until 2026-05-28 08:22:24.525357284 +0000 UTC m=+622.961536431 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-offline-tls") pod "feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" (UID: "22332100-a3b6-45c7-acd0-3a5c690c4ff3") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 08:22:24.025718 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.025696    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22332100-a3b6-45c7-acd0-3a5c690c4ff3-feast-data\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.025791 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.025771    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/22332100-a3b6-45c7-acd0-3a5c690c4ff3-registry-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.027813 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.027790    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-online-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.042089 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.042065    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-rd2fz\" (UniqueName: \"kubernetes.io/projected/22332100-a3b6-45c7-acd0-3a5c690c4ff3-kube-api-access-rd2fz\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.530820 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.530779    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-ui-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.531005 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.530832    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-offline-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.533275 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.533244    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-ui-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.533383 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.533279    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-offline-tls\") pod \"feast-simple-feast-remote-setup-7966b9ccbb-zdk4v\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.763975 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.763915    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:24.896272 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:24.896248    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"]
May 28 08:22:24.897457 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:22:24.897430    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod22332100_a3b6_45c7_acd0_3a5c690c4ff3.slice/crio-1c99dd2912b161159eebb22f3981f7040193b5ba19da0ca26c1990693b66715d WatchSource:0}: Error finding container 1c99dd2912b161159eebb22f3981f7040193b5ba19da0ca26c1990693b66715d: Status 404 returned error can't find the container with id 1c99dd2912b161159eebb22f3981f7040193b5ba19da0ca26c1990693b66715d
May 28 08:22:25.226251 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:25.226156    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" event={"ID":"22332100-a3b6-45c7-acd0-3a5c690c4ff3","Type":"ContainerStarted","Data":"f3e710044c31f207529c6f68f6222c869c70481ab9342c58502a84ddaf9b3c95"}
May 28 08:22:25.226251 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:25.226195    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" event={"ID":"22332100-a3b6-45c7-acd0-3a5c690c4ff3","Type":"ContainerStarted","Data":"1c99dd2912b161159eebb22f3981f7040193b5ba19da0ca26c1990693b66715d"}
May 28 08:22:30.243378 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:30.243325    2577 generic.go:358] "Generic (PLEG): container finished" podID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerID="f3e710044c31f207529c6f68f6222c869c70481ab9342c58502a84ddaf9b3c95" exitCode=0
May 28 08:22:30.243788 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:30.243396    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" event={"ID":"22332100-a3b6-45c7-acd0-3a5c690c4ff3","Type":"ContainerDied","Data":"f3e710044c31f207529c6f68f6222c869c70481ab9342c58502a84ddaf9b3c95"}
May 28 08:22:31.248783 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:31.248749    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" event={"ID":"22332100-a3b6-45c7-acd0-3a5c690c4ff3","Type":"ContainerStarted","Data":"35df087e002c71fb0e8e9c62a1b43c3e1ed4ea69decaa5c9c1f5e3af96676e29"}
May 28 08:22:36.265339 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:36.265306    2577 generic.go:358] "Generic (PLEG): container finished" podID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerID="35df087e002c71fb0e8e9c62a1b43c3e1ed4ea69decaa5c9c1f5e3af96676e29" exitCode=0
May 28 08:22:36.265771 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:36.265378    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" event={"ID":"22332100-a3b6-45c7-acd0-3a5c690c4ff3","Type":"ContainerDied","Data":"35df087e002c71fb0e8e9c62a1b43c3e1ed4ea69decaa5c9c1f5e3af96676e29"}
May 28 08:22:37.272401 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:37.272362    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" event={"ID":"22332100-a3b6-45c7-acd0-3a5c690c4ff3","Type":"ContainerStarted","Data":"91fbfaf939bdad7e01eb239224fb3ebca3add7fa6b495ff839c20f6f65ec062b"}
May 28 08:22:37.272941 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:37.272412    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" event={"ID":"22332100-a3b6-45c7-acd0-3a5c690c4ff3","Type":"ContainerStarted","Data":"d057fb3227e3cc9fabba647c14d9ddb7bb2ad243ca07a3012aa3338e9160eb1b"}
May 28 08:22:37.272941 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:37.272424    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" event={"ID":"22332100-a3b6-45c7-acd0-3a5c690c4ff3","Type":"ContainerStarted","Data":"9da020cc690d826de45a2d2a476564e6447e92ebc3f1d2b291c6b4f61fef5cc6"}
May 28 08:22:37.299647 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:37.299584    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podStartSLOduration=14.2995694 podStartE2EDuration="14.2995694s" podCreationTimestamp="2026-05-28 08:22:23 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:22:37.29703642 +0000 UTC m=+635.733215624" watchObservedRunningTime="2026-05-28 08:22:37.2995694 +0000 UTC m=+635.735748566"
May 28 08:22:39.764153 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:39.764110    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:39.764637 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:39.764165    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:39.764637 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:39.764180    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:39.765951 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:39.765916    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.26:8816: connect: connection refused"
May 28 08:22:39.765951 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:39.765932    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 28 08:22:39.766110 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:39.765968    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 28 08:22:42.765642 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:42.765610    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:42.768395 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:42.765755    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:42.768395 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:42.766081    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:42.768395 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:42.766113    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:42.768395 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:42.766380    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:42.768395 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:42.766484    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:42.769430 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:42.769415    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:43.294611 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:43.294579    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:22:43.297626 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:22:43.297602    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:23:45.104044 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:45.104007    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"]
May 28 08:23:45.104523 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:45.104445    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="online" containerID="cri-o://9da020cc690d826de45a2d2a476564e6447e92ebc3f1d2b291c6b4f61fef5cc6" gracePeriod=30
May 28 08:23:45.104600 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:45.104506    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="ui" containerID="cri-o://91fbfaf939bdad7e01eb239224fb3ebca3add7fa6b495ff839c20f6f65ec062b" gracePeriod=30
May 28 08:23:45.104600 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:45.104585    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="offline" containerID="cri-o://d057fb3227e3cc9fabba647c14d9ddb7bb2ad243ca07a3012aa3338e9160eb1b" gracePeriod=30
May 28 08:23:45.229108 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:45.229072    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"]
May 28 08:23:45.229441 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:45.229414    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="registry" containerID="cri-o://4b2951fd0458c662ddc19a46a055704fe7b83eebd1d7f2e630af1dd28d795ef6" gracePeriod=30
May 28 08:23:45.229608 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:45.229584    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="offline" containerID="cri-o://7d49d166a6c02504239613fcd7f8d80f7e24b3e15783cc5b24795af3df584475" gracePeriod=30
May 28 08:23:45.229754 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:45.229499    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="online" containerID="cri-o://92d023dd6c55e222c80e6b6f897c116260ae69de2007741b1732216ca7e608dd" gracePeriod=30
May 28 08:23:45.229754 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:45.229476    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="ui" containerID="cri-o://e3cb7eb70438104149af8e6549eb13898db09fc43a3bb44c718c7300a7491c64" gracePeriod=30
May 28 08:23:46.495082 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:46.495049    2577 generic.go:358] "Generic (PLEG): container finished" podID="8d915380-4306-4285-8803-95bbec28d0a7" containerID="e3cb7eb70438104149af8e6549eb13898db09fc43a3bb44c718c7300a7491c64" exitCode=0
May 28 08:23:46.495082 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:46.495077    2577 generic.go:358] "Generic (PLEG): container finished" podID="8d915380-4306-4285-8803-95bbec28d0a7" containerID="92d023dd6c55e222c80e6b6f897c116260ae69de2007741b1732216ca7e608dd" exitCode=0
May 28 08:23:46.495606 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:46.495130    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerDied","Data":"e3cb7eb70438104149af8e6549eb13898db09fc43a3bb44c718c7300a7491c64"}
May 28 08:23:46.495606 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:46.495173    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerDied","Data":"92d023dd6c55e222c80e6b6f897c116260ae69de2007741b1732216ca7e608dd"}
May 28 08:23:46.497301 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:46.497278    2577 generic.go:358] "Generic (PLEG): container finished" podID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerID="91fbfaf939bdad7e01eb239224fb3ebca3add7fa6b495ff839c20f6f65ec062b" exitCode=0
May 28 08:23:46.497301 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:46.497297    2577 generic.go:358] "Generic (PLEG): container finished" podID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerID="9da020cc690d826de45a2d2a476564e6447e92ebc3f1d2b291c6b4f61fef5cc6" exitCode=0
May 28 08:23:46.497435 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:46.497320    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" event={"ID":"22332100-a3b6-45c7-acd0-3a5c690c4ff3","Type":"ContainerDied","Data":"91fbfaf939bdad7e01eb239224fb3ebca3add7fa6b495ff839c20f6f65ec062b"}
May 28 08:23:46.497435 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:46.497338    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" event={"ID":"22332100-a3b6-45c7-acd0-3a5c690c4ff3","Type":"ContainerDied","Data":"9da020cc690d826de45a2d2a476564e6447e92ebc3f1d2b291c6b4f61fef5cc6"}
May 28 08:23:51.017049 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:51.016941    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 28 08:23:51.033985 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:51.033949    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 28 08:23:52.766367 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:52.766317    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 28 08:23:53.294771 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:23:53.294738    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 28 08:24:01.017178 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:01.017129    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 28 08:24:01.034473 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:01.034434    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 28 08:24:02.766757 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:02.766712    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 28 08:24:03.295727 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:03.295684    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 28 08:24:11.016685 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:11.016619    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.25:8443: connect: connection refused"
May 28 08:24:11.017104 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:11.016812    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:24:11.033999 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:11.033960    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="online" probeResult="failure" output="Get \"https://10.134.0.25:6567/health\": dial tcp 10.134.0.25:6567: connect: connection refused"
May 28 08:24:11.034140 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:11.034052    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:24:12.766768 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:12.766730    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.26:8443: connect: connection refused"
May 28 08:24:12.767187 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:12.766856    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:24:13.295433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:13.295385    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.26:6567/health\": dial tcp 10.134.0.26:6567: connect: connection refused"
May 28 08:24:13.295608 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:13.295546    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:24:15.584241 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.584209    2577 generic.go:358] "Generic (PLEG): container finished" podID="8d915380-4306-4285-8803-95bbec28d0a7" containerID="7d49d166a6c02504239613fcd7f8d80f7e24b3e15783cc5b24795af3df584475" exitCode=137
May 28 08:24:15.584241 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.584234    2577 generic.go:358] "Generic (PLEG): container finished" podID="8d915380-4306-4285-8803-95bbec28d0a7" containerID="4b2951fd0458c662ddc19a46a055704fe7b83eebd1d7f2e630af1dd28d795ef6" exitCode=137
May 28 08:24:15.584709 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.584286    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerDied","Data":"7d49d166a6c02504239613fcd7f8d80f7e24b3e15783cc5b24795af3df584475"}
May 28 08:24:15.584709 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.584319    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerDied","Data":"4b2951fd0458c662ddc19a46a055704fe7b83eebd1d7f2e630af1dd28d795ef6"}
May 28 08:24:15.586379 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.586357    2577 generic.go:358] "Generic (PLEG): container finished" podID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerID="d057fb3227e3cc9fabba647c14d9ddb7bb2ad243ca07a3012aa3338e9160eb1b" exitCode=137
May 28 08:24:15.586481 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.586439    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" event={"ID":"22332100-a3b6-45c7-acd0-3a5c690c4ff3","Type":"ContainerDied","Data":"d057fb3227e3cc9fabba647c14d9ddb7bb2ad243ca07a3012aa3338e9160eb1b"}
May 28 08:24:15.756211 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.756186    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:24:15.877081 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.877061    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:24:15.884968 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.884943    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-online-tls\") pod \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") "
May 28 08:24:15.885062 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.884993    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-ui-tls\") pod \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") "
May 28 08:24:15.885062 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.885049    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-rd2fz\" (UniqueName: \"kubernetes.io/projected/22332100-a3b6-45c7-acd0-3a5c690c4ff3-kube-api-access-rd2fz\") pod \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") "
May 28 08:24:15.885131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.885068    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/22332100-a3b6-45c7-acd0-3a5c690c4ff3-registry-tls\") pod \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") "
May 28 08:24:15.885131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.885096    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22332100-a3b6-45c7-acd0-3a5c690c4ff3-feast-data\") pod \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") "
May 28 08:24:15.885131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.885120    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-offline-tls\") pod \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\" (UID: \"22332100-a3b6-45c7-acd0-3a5c690c4ff3\") "
May 28 08:24:15.885464 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.885433    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/22332100-a3b6-45c7-acd0-3a5c690c4ff3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "22332100-a3b6-45c7-acd0-3a5c690c4ff3" (UID: "22332100-a3b6-45c7-acd0-3a5c690c4ff3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:24:15.885628 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.885575    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/22332100-a3b6-45c7-acd0-3a5c690c4ff3-feast-data" (OuterVolumeSpecName: "feast-data") pod "22332100-a3b6-45c7-acd0-3a5c690c4ff3" (UID: "22332100-a3b6-45c7-acd0-3a5c690c4ff3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:24:15.887246 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.887222    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/22332100-a3b6-45c7-acd0-3a5c690c4ff3-kube-api-access-rd2fz" (OuterVolumeSpecName: "kube-api-access-rd2fz") pod "22332100-a3b6-45c7-acd0-3a5c690c4ff3" (UID: "22332100-a3b6-45c7-acd0-3a5c690c4ff3"). InnerVolumeSpecName "kube-api-access-rd2fz". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:24:15.887332 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.887251    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-online-tls" (OuterVolumeSpecName: "online-tls") pod "22332100-a3b6-45c7-acd0-3a5c690c4ff3" (UID: "22332100-a3b6-45c7-acd0-3a5c690c4ff3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:24:15.887332 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.887264    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "22332100-a3b6-45c7-acd0-3a5c690c4ff3" (UID: "22332100-a3b6-45c7-acd0-3a5c690c4ff3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:24:15.887419 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.887336    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "22332100-a3b6-45c7-acd0-3a5c690c4ff3" (UID: "22332100-a3b6-45c7-acd0-3a5c690c4ff3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:24:15.985847 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.985814    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-offline-tls\") pod \"8d915380-4306-4285-8803-95bbec28d0a7\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") "
May 28 08:24:15.985847 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.985862    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-ui-tls\") pod \"8d915380-4306-4285-8803-95bbec28d0a7\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") "
May 28 08:24:15.986117 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.985883    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-online-tls\") pod \"8d915380-4306-4285-8803-95bbec28d0a7\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") "
May 28 08:24:15.986117 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.985910    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d915380-4306-4285-8803-95bbec28d0a7-feast-data\") pod \"8d915380-4306-4285-8803-95bbec28d0a7\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") "
May 28 08:24:15.986117 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.985956    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-w9pqb\" (UniqueName: \"kubernetes.io/projected/8d915380-4306-4285-8803-95bbec28d0a7-kube-api-access-w9pqb\") pod \"8d915380-4306-4285-8803-95bbec28d0a7\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") "
May 28 08:24:15.986117 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.985993    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-registry-tls\") pod \"8d915380-4306-4285-8803-95bbec28d0a7\" (UID: \"8d915380-4306-4285-8803-95bbec28d0a7\") "
May 28 08:24:15.986314 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.986222    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-rd2fz\" (UniqueName: \"kubernetes.io/projected/22332100-a3b6-45c7-acd0-3a5c690c4ff3-kube-api-access-rd2fz\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:24:15.986314 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.986241    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/22332100-a3b6-45c7-acd0-3a5c690c4ff3-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:24:15.986314 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.986256    2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/22332100-a3b6-45c7-acd0-3a5c690c4ff3-feast-data\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:24:15.986314 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.986268    2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-offline-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:24:15.986314 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.986281    2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-online-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:24:15.986314 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.986293    2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/22332100-a3b6-45c7-acd0-3a5c690c4ff3-ui-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:24:15.986568 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.986539    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/8d915380-4306-4285-8803-95bbec28d0a7-feast-data" (OuterVolumeSpecName: "feast-data") pod "8d915380-4306-4285-8803-95bbec28d0a7" (UID: "8d915380-4306-4285-8803-95bbec28d0a7"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:24:15.988418 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.988382    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "8d915380-4306-4285-8803-95bbec28d0a7" (UID: "8d915380-4306-4285-8803-95bbec28d0a7"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:24:15.988615 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.988586    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-online-tls" (OuterVolumeSpecName: "online-tls") pod "8d915380-4306-4285-8803-95bbec28d0a7" (UID: "8d915380-4306-4285-8803-95bbec28d0a7"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:24:15.988615 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.988607    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "8d915380-4306-4285-8803-95bbec28d0a7" (UID: "8d915380-4306-4285-8803-95bbec28d0a7"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:24:15.988615 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.988624    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/8d915380-4306-4285-8803-95bbec28d0a7-kube-api-access-w9pqb" (OuterVolumeSpecName: "kube-api-access-w9pqb") pod "8d915380-4306-4285-8803-95bbec28d0a7" (UID: "8d915380-4306-4285-8803-95bbec28d0a7"). InnerVolumeSpecName "kube-api-access-w9pqb". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:24:15.988826 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:15.988687    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "8d915380-4306-4285-8803-95bbec28d0a7" (UID: "8d915380-4306-4285-8803-95bbec28d0a7"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:24:16.087058 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.087008    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-w9pqb\" (UniqueName: \"kubernetes.io/projected/8d915380-4306-4285-8803-95bbec28d0a7-kube-api-access-w9pqb\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:24:16.087058 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.087057    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:24:16.087276 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.087088    2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-offline-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:24:16.087276 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.087101    2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-ui-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:24:16.087276 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.087111    2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/8d915380-4306-4285-8803-95bbec28d0a7-online-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:24:16.087276 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.087122    2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/8d915380-4306-4285-8803-95bbec28d0a7-feast-data\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:24:16.591597 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.591562    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx" event={"ID":"8d915380-4306-4285-8803-95bbec28d0a7","Type":"ContainerDied","Data":"f58e6f509851ac42e270eae3644f9c61e0a09eb54f823f3a6152e73b1db80011"}
May 28 08:24:16.591597 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.591597    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"
May 28 08:24:16.592220 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.591618    2577 scope.go:117] "RemoveContainer" containerID="e3cb7eb70438104149af8e6549eb13898db09fc43a3bb44c718c7300a7491c64"
May 28 08:24:16.594110 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.594083    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v" event={"ID":"22332100-a3b6-45c7-acd0-3a5c690c4ff3","Type":"ContainerDied","Data":"1c99dd2912b161159eebb22f3981f7040193b5ba19da0ca26c1990693b66715d"}
May 28 08:24:16.594234 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.594172    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"
May 28 08:24:16.600471 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.600451    2577 scope.go:117] "RemoveContainer" containerID="7d49d166a6c02504239613fcd7f8d80f7e24b3e15783cc5b24795af3df584475"
May 28 08:24:16.608125 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.608103    2577 scope.go:117] "RemoveContainer" containerID="92d023dd6c55e222c80e6b6f897c116260ae69de2007741b1732216ca7e608dd"
May 28 08:24:16.615336 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.615314    2577 scope.go:117] "RemoveContainer" containerID="4b2951fd0458c662ddc19a46a055704fe7b83eebd1d7f2e630af1dd28d795ef6"
May 28 08:24:16.621693 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.621649    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"]
May 28 08:24:16.623034 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.623017    2577 scope.go:117] "RemoveContainer" containerID="dd59ca99cc0431bfe20cacca1afeb429b92edf60f4b82ef8187b02e6d4be2c9b"
May 28 08:24:16.629525 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.629504    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-7966b9ccbb-zdk4v"]
May 28 08:24:16.630476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.630462    2577 scope.go:117] "RemoveContainer" containerID="3ac972b90a9ea4ad167ecd3bd0c42a2ac0dac2a6a67129def44de178d9935c98"
May 28 08:24:16.637667 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.637639    2577 scope.go:117] "RemoveContainer" containerID="91fbfaf939bdad7e01eb239224fb3ebca3add7fa6b495ff839c20f6f65ec062b"
May 28 08:24:16.642411 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.642390    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"]
May 28 08:24:16.646308 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.646284    2577 scope.go:117] "RemoveContainer" containerID="d057fb3227e3cc9fabba647c14d9ddb7bb2ad243ca07a3012aa3338e9160eb1b"
May 28 08:24:16.647621 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.647545    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-5b6596f8cb-4p4xx"]
May 28 08:24:16.653521 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.653509    2577 scope.go:117] "RemoveContainer" containerID="9da020cc690d826de45a2d2a476564e6447e92ebc3f1d2b291c6b4f61fef5cc6"
May 28 08:24:16.660642 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.660613    2577 scope.go:117] "RemoveContainer" containerID="35df087e002c71fb0e8e9c62a1b43c3e1ed4ea69decaa5c9c1f5e3af96676e29"
May 28 08:24:16.667421 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:16.667381    2577 scope.go:117] "RemoveContainer" containerID="f3e710044c31f207529c6f68f6222c869c70481ab9342c58502a84ddaf9b3c95"
May 28 08:24:18.242022 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:18.241991    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" path="/var/lib/kubelet/pods/22332100-a3b6-45c7-acd0-3a5c690c4ff3/volumes"
May 28 08:24:18.242518 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:18.242504    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="8d915380-4306-4285-8803-95bbec28d0a7" path="/var/lib/kubelet/pods/8d915380-4306-4285-8803-95bbec28d0a7/volumes"
May 28 08:24:28.242772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.242736    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-z92nz"]
May 28 08:24:28.243230 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243179    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="registry"
May 28 08:24:28.243230 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243198    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="registry"
May 28 08:24:28.243230 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243212    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="feast-apply"
May 28 08:24:28.243230 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243221    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="feast-apply"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243237    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="online"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243246    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="online"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243257    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="ui"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243265    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="ui"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243278    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="feast-apply"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243287    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="feast-apply"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243298    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="online"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243306    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="online"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243318    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="feast-init"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243327    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="feast-init"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243348    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="ui"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243357    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="ui"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243367    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="feast-init"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243375    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="feast-init"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243394    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="offline"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243404    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="offline"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243417    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="offline"
May 28 08:24:28.243454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243425    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="offline"
May 28 08:24:28.244371 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243496    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="ui"
May 28 08:24:28.244371 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243509    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="ui"
May 28 08:24:28.244371 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243521    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="registry"
May 28 08:24:28.244371 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243534    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="offline"
May 28 08:24:28.244371 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243544    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="offline"
May 28 08:24:28.244371 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243553    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="22332100-a3b6-45c7-acd0-3a5c690c4ff3" containerName="online"
May 28 08:24:28.244371 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.243565    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="8d915380-4306-4285-8803-95bbec28d0a7" containerName="online"
May 28 08:24:28.246651 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.246628    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-z92nz"
May 28 08:24:28.250285 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.250050    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"default-dockercfg-n27cx\""
May 28 08:24:28.250285 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.250050    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 08:24:28.250285 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.250050    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"postgres-secret\""
May 28 08:24:28.250521 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.250305    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 08:24:28.250521 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.250474    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-z92nz"]
May 28 08:24:28.281762 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.281729    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ad953d03-5213-4b68-8492-3c7037db2a76-postgresdata\") pod \"postgres-7c7656cc6d-z92nz\" (UID: \"ad953d03-5213-4b68-8492-3c7037db2a76\") " pod="test-ns-feast/postgres-7c7656cc6d-z92nz"
May 28 08:24:28.281950 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.281792    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8qtw5\" (UniqueName: \"kubernetes.io/projected/ad953d03-5213-4b68-8492-3c7037db2a76-kube-api-access-8qtw5\") pod \"postgres-7c7656cc6d-z92nz\" (UID: \"ad953d03-5213-4b68-8492-3c7037db2a76\") " pod="test-ns-feast/postgres-7c7656cc6d-z92nz"
May 28 08:24:28.382921 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.382882    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8qtw5\" (UniqueName: \"kubernetes.io/projected/ad953d03-5213-4b68-8492-3c7037db2a76-kube-api-access-8qtw5\") pod \"postgres-7c7656cc6d-z92nz\" (UID: \"ad953d03-5213-4b68-8492-3c7037db2a76\") " pod="test-ns-feast/postgres-7c7656cc6d-z92nz"
May 28 08:24:28.383112 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.382942    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ad953d03-5213-4b68-8492-3c7037db2a76-postgresdata\") pod \"postgres-7c7656cc6d-z92nz\" (UID: \"ad953d03-5213-4b68-8492-3c7037db2a76\") " pod="test-ns-feast/postgres-7c7656cc6d-z92nz"
May 28 08:24:28.383262 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.383246    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ad953d03-5213-4b68-8492-3c7037db2a76-postgresdata\") pod \"postgres-7c7656cc6d-z92nz\" (UID: \"ad953d03-5213-4b68-8492-3c7037db2a76\") " pod="test-ns-feast/postgres-7c7656cc6d-z92nz"
May 28 08:24:28.392932 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.392913    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8qtw5\" (UniqueName: \"kubernetes.io/projected/ad953d03-5213-4b68-8492-3c7037db2a76-kube-api-access-8qtw5\") pod \"postgres-7c7656cc6d-z92nz\" (UID: \"ad953d03-5213-4b68-8492-3c7037db2a76\") " pod="test-ns-feast/postgres-7c7656cc6d-z92nz"
May 28 08:24:28.558215 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.558174    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-z92nz"
May 28 08:24:28.684480 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.684456    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-z92nz"]
May 28 08:24:28.686621 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:24:28.686588    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podad953d03_5213_4b68_8492_3c7037db2a76.slice/crio-32cdf83dfe7106180bb6946a5349c57f4db7888331b1915332d8e40b5b8dd0ea WatchSource:0}: Error finding container 32cdf83dfe7106180bb6946a5349c57f4db7888331b1915332d8e40b5b8dd0ea: Status 404 returned error can't find the container with id 32cdf83dfe7106180bb6946a5349c57f4db7888331b1915332d8e40b5b8dd0ea
May 28 08:24:28.688371 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:28.688356    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 08:24:29.635335 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:29.635296    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-z92nz" event={"ID":"ad953d03-5213-4b68-8492-3c7037db2a76","Type":"ContainerStarted","Data":"32cdf83dfe7106180bb6946a5349c57f4db7888331b1915332d8e40b5b8dd0ea"}
May 28 08:24:34.651384 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:34.651295    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-z92nz" event={"ID":"ad953d03-5213-4b68-8492-3c7037db2a76","Type":"ContainerStarted","Data":"3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19"}
May 28 08:24:34.671650 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:34.671373    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/postgres-7c7656cc6d-z92nz" podStartSLOduration=0.965789599 podStartE2EDuration="6.671355593s" podCreationTimestamp="2026-05-28 08:24:28 +0000 UTC" firstStartedPulling="2026-05-28 08:24:28.688517834 +0000 UTC m=+747.124696978" lastFinishedPulling="2026-05-28 08:24:34.394083824 +0000 UTC m=+752.830262972" observedRunningTime="2026-05-28 08:24:34.67087033 +0000 UTC m=+753.107049499" watchObservedRunningTime="2026-05-28 08:24:34.671355593 +0000 UTC m=+753.107534753"
May 28 08:24:38.937549 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.937514    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"]
May 28 08:24:38.944470 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.944441    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:38.947463 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.947438    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-online-tls\""
May 28 08:24:38.947463 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.947468    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-dockercfg-6hdnr\""
May 28 08:24:38.947735 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.947708    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-registry-tls\""
May 28 08:24:38.947735 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.947727    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-offline-tls\""
May 28 08:24:38.947985 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.947967    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"feast-credit-scoring-intra-comm\""
May 28 08:24:38.952114 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.952083    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"]
May 28 08:24:38.972285 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.972254    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-online-tls\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:38.972444 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.972301    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ebd442f7-6104-4efe-a863-647fea9cfd5a-feast-data\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:38.972444 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.972383    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-8tl6r\" (UniqueName: \"kubernetes.io/projected/ebd442f7-6104-4efe-a863-647fea9cfd5a-kube-api-access-8tl6r\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:38.972444 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.972413    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-offline-tls\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:38.972602 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:38.972522    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-registry-tls\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.073255 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.073217    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-online-tls\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.073255 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.073256    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ebd442f7-6104-4efe-a863-647fea9cfd5a-feast-data\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.073517 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.073289    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-8tl6r\" (UniqueName: \"kubernetes.io/projected/ebd442f7-6104-4efe-a863-647fea9cfd5a-kube-api-access-8tl6r\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.073517 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:24:39.073383    2577 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found
May 28 08:24:39.073517 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.073400    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-offline-tls\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.073517 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:24:39.073466    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-online-tls podName:ebd442f7-6104-4efe-a863-647fea9cfd5a nodeName:}" failed. No retries permitted until 2026-05-28 08:24:39.57344476 +0000 UTC m=+758.009623909 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-online-tls") pod "feast-credit-scoring-76fbfc7469-nx8rs" (UID: "ebd442f7-6104-4efe-a863-647fea9cfd5a") : secret "feast-credit-scoring-online-tls" not found
May 28 08:24:39.073517 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:24:39.073511    2577 secret.go:189] Couldn't get secret test-ns-feast/feast-credit-scoring-offline-tls: secret "feast-credit-scoring-offline-tls" not found
May 28 08:24:39.073793 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.073532    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-registry-tls\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.073793 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:24:39.073598    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-offline-tls podName:ebd442f7-6104-4efe-a863-647fea9cfd5a nodeName:}" failed. No retries permitted until 2026-05-28 08:24:39.573578558 +0000 UTC m=+758.009757703 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-offline-tls") pod "feast-credit-scoring-76fbfc7469-nx8rs" (UID: "ebd442f7-6104-4efe-a863-647fea9cfd5a") : secret "feast-credit-scoring-offline-tls" not found
May 28 08:24:39.073793 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.073640    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ebd442f7-6104-4efe-a863-647fea9cfd5a-feast-data\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.076211 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.076191    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-registry-tls\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.084629 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.084606    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-8tl6r\" (UniqueName: \"kubernetes.io/projected/ebd442f7-6104-4efe-a863-647fea9cfd5a-kube-api-access-8tl6r\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.579390 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.579353    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-offline-tls\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.579621 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.579434    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-online-tls\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.581987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.581956    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-offline-tls\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.582109 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.582029    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-online-tls\") pod \"feast-credit-scoring-76fbfc7469-nx8rs\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") " pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:39.857466 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:39.857431    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:40.001400 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:40.001369    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"]
May 28 08:24:40.003430 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:24:40.003403    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-podebd442f7_6104_4efe_a863_647fea9cfd5a.slice/crio-55d187d660e6fa5dd5e030d3c42d85258444f33691ebcb2bb0e4c0ecf47d86e7 WatchSource:0}: Error finding container 55d187d660e6fa5dd5e030d3c42d85258444f33691ebcb2bb0e4c0ecf47d86e7: Status 404 returned error can't find the container with id 55d187d660e6fa5dd5e030d3c42d85258444f33691ebcb2bb0e4c0ecf47d86e7
May 28 08:24:40.670029 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:40.669986    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" event={"ID":"ebd442f7-6104-4efe-a863-647fea9cfd5a","Type":"ContainerStarted","Data":"64a7e22f5080653808594df79a4ba7d6248011566569716859a3e53094918a01"}
May 28 08:24:40.670029 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:40.670029    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" event={"ID":"ebd442f7-6104-4efe-a863-647fea9cfd5a","Type":"ContainerStarted","Data":"55d187d660e6fa5dd5e030d3c42d85258444f33691ebcb2bb0e4c0ecf47d86e7"}
May 28 08:24:41.674352 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:41.674311    2577 generic.go:358] "Generic (PLEG): container finished" podID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerID="64a7e22f5080653808594df79a4ba7d6248011566569716859a3e53094918a01" exitCode=0
May 28 08:24:41.674798 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:41.674398    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" event={"ID":"ebd442f7-6104-4efe-a863-647fea9cfd5a","Type":"ContainerDied","Data":"64a7e22f5080653808594df79a4ba7d6248011566569716859a3e53094918a01"}
May 28 08:24:42.679951 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:42.679914    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" event={"ID":"ebd442f7-6104-4efe-a863-647fea9cfd5a","Type":"ContainerStarted","Data":"b0f360d33dd44c3b4b87525a7659fa0ded0e9c82f3e1307c1333402c5c270483"}
May 28 08:24:48.699866 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:48.699833    2577 generic.go:358] "Generic (PLEG): container finished" podID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerID="b0f360d33dd44c3b4b87525a7659fa0ded0e9c82f3e1307c1333402c5c270483" exitCode=0
May 28 08:24:48.700247 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:48.699872    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" event={"ID":"ebd442f7-6104-4efe-a863-647fea9cfd5a","Type":"ContainerDied","Data":"b0f360d33dd44c3b4b87525a7659fa0ded0e9c82f3e1307c1333402c5c270483"}
May 28 08:24:49.707374 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:49.707324    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" event={"ID":"ebd442f7-6104-4efe-a863-647fea9cfd5a","Type":"ContainerStarted","Data":"5dcf3b54e1626123c3192051c7055b07193be833856f3e6d00ee565d1ffd5fd2"}
May 28 08:24:49.707843 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:49.707387    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" event={"ID":"ebd442f7-6104-4efe-a863-647fea9cfd5a","Type":"ContainerStarted","Data":"a6fa772cfada4315c85c621eb5ff6565d4f9af6738f590523a2cf63f5d2ddb69"}
May 28 08:24:49.707843 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:49.707403    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" event={"ID":"ebd442f7-6104-4efe-a863-647fea9cfd5a","Type":"ContainerStarted","Data":"fe2a81314661eb640cde659e6fa0331b167626b7015172931a84c399e0219ccd"}
May 28 08:24:49.742732 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:49.742644    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" podStartSLOduration=11.742624297999999 podStartE2EDuration="11.742624298s" podCreationTimestamp="2026-05-28 08:24:38 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:24:49.740204619 +0000 UTC m=+768.176383798" watchObservedRunningTime="2026-05-28 08:24:49.742624298 +0000 UTC m=+768.178803466"
May 28 08:24:49.858529 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:49.858490    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:49.858718 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:49.858545    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:49.858718 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:49.858560    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:51.858688 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:51.858631    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:51.859269 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:51.859242    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:51.859269 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:51.859271    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:51.860163 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:51.860127    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused"
May 28 08:24:51.860282 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:51.860154    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.28:8816: connect: connection refused"
May 28 08:24:51.860421 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:51.860391    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 28 08:24:54.858466 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:54.858404    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused"
May 28 08:24:54.859989 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:54.859951    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:54.860822 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:54.860776    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:54.864074 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:54.864054    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:55.730777 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:55.730746    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:24:57.858180 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:24:57.858143    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.28:6571: connect: connection refused"
May 28 08:25:00.858905 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:25:00.858874    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:25:00.859607 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:25:00.859589    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:26:05.665636 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.665603    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"]
May 28 08:26:05.666092 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.665960    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="registry" containerID="cri-o://fe2a81314661eb640cde659e6fa0331b167626b7015172931a84c399e0219ccd" gracePeriod=30
May 28 08:26:05.666092 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.666017    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="online" containerID="cri-o://a6fa772cfada4315c85c621eb5ff6565d4f9af6738f590523a2cf63f5d2ddb69" gracePeriod=30
May 28 08:26:05.666194 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.666018    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="offline" containerID="cri-o://5dcf3b54e1626123c3192051c7055b07193be833856f3e6d00ee565d1ffd5fd2" gracePeriod=30
May 28 08:26:05.686798 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.686762    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-z92nz"]
May 28 08:26:05.686995 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.686971    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/postgres-7c7656cc6d-z92nz" podUID="ad953d03-5213-4b68-8492-3c7037db2a76" containerName="postgres" containerID="cri-o://3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19" gracePeriod=30
May 28 08:26:05.928180 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.928158    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-z92nz"
May 28 08:26:05.939118 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.939086    2577 generic.go:358] "Generic (PLEG): container finished" podID="ad953d03-5213-4b68-8492-3c7037db2a76" containerID="3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19" exitCode=0
May 28 08:26:05.939256 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.939130    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/postgres-7c7656cc6d-z92nz"
May 28 08:26:05.939256 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.939181    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-z92nz" event={"ID":"ad953d03-5213-4b68-8492-3c7037db2a76","Type":"ContainerDied","Data":"3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19"}
May 28 08:26:05.939256 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.939227    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/postgres-7c7656cc6d-z92nz" event={"ID":"ad953d03-5213-4b68-8492-3c7037db2a76","Type":"ContainerDied","Data":"32cdf83dfe7106180bb6946a5349c57f4db7888331b1915332d8e40b5b8dd0ea"}
May 28 08:26:05.939256 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.939249    2577 scope.go:117] "RemoveContainer" containerID="3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19"
May 28 08:26:05.948355 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.948330    2577 scope.go:117] "RemoveContainer" containerID="3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19"
May 28 08:26:05.948704 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:26:05.948677    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19\": container with ID starting with 3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19 not found: ID does not exist" containerID="3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19"
May 28 08:26:05.948808 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:05.948716    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19"} err="failed to get container status \"3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19\": rpc error: code = NotFound desc = could not find container \"3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19\": container with ID starting with 3e2bbfa8db6cf3224cffe1e7b056943b54891fe408903d0bb1241480bcb6aa19 not found: ID does not exist"
May 28 08:26:06.013972 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:06.013943    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8qtw5\" (UniqueName: \"kubernetes.io/projected/ad953d03-5213-4b68-8492-3c7037db2a76-kube-api-access-8qtw5\") pod \"ad953d03-5213-4b68-8492-3c7037db2a76\" (UID: \"ad953d03-5213-4b68-8492-3c7037db2a76\") "
May 28 08:26:06.014123 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:06.014009    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ad953d03-5213-4b68-8492-3c7037db2a76-postgresdata\") pod \"ad953d03-5213-4b68-8492-3c7037db2a76\" (UID: \"ad953d03-5213-4b68-8492-3c7037db2a76\") "
May 28 08:26:06.016162 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:06.016128    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ad953d03-5213-4b68-8492-3c7037db2a76-kube-api-access-8qtw5" (OuterVolumeSpecName: "kube-api-access-8qtw5") pod "ad953d03-5213-4b68-8492-3c7037db2a76" (UID: "ad953d03-5213-4b68-8492-3c7037db2a76"). InnerVolumeSpecName "kube-api-access-8qtw5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:26:06.043331 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:06.043291    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ad953d03-5213-4b68-8492-3c7037db2a76-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "ad953d03-5213-4b68-8492-3c7037db2a76" (UID: "ad953d03-5213-4b68-8492-3c7037db2a76"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:26:06.115549 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:06.115509    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8qtw5\" (UniqueName: \"kubernetes.io/projected/ad953d03-5213-4b68-8492-3c7037db2a76-kube-api-access-8qtw5\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:26:06.115549 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:06.115543    2577 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/ad953d03-5213-4b68-8492-3c7037db2a76-postgresdata\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:26:06.264413 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:06.264378    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-z92nz"]
May 28 08:26:06.268305 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:06.268279    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/postgres-7c7656cc6d-z92nz"]
May 28 08:26:06.944503 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:06.944466    2577 generic.go:358] "Generic (PLEG): container finished" podID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerID="a6fa772cfada4315c85c621eb5ff6565d4f9af6738f590523a2cf63f5d2ddb69" exitCode=0
May 28 08:26:06.944900 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:06.944541    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" event={"ID":"ebd442f7-6104-4efe-a863-647fea9cfd5a","Type":"ContainerDied","Data":"a6fa772cfada4315c85c621eb5ff6565d4f9af6738f590523a2cf63f5d2ddb69"}
May 28 08:26:08.241629 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:08.241594    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ad953d03-5213-4b68-8492-3c7037db2a76" path="/var/lib/kubelet/pods/ad953d03-5213-4b68-8492-3c7037db2a76/volumes"
May 28 08:26:15.727921 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:15.727870    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 28 08:26:25.728303 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:25.728256    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 28 08:26:35.728005 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:35.727964    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.28:6567/health\": dial tcp 10.134.0.28:6567: connect: connection refused"
May 28 08:26:35.728337 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:35.728110    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:26:36.032594 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.032563    2577 generic.go:358] "Generic (PLEG): container finished" podID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerID="5dcf3b54e1626123c3192051c7055b07193be833856f3e6d00ee565d1ffd5fd2" exitCode=137
May 28 08:26:36.032594 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.032589    2577 generic.go:358] "Generic (PLEG): container finished" podID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerID="fe2a81314661eb640cde659e6fa0331b167626b7015172931a84c399e0219ccd" exitCode=137
May 28 08:26:36.032813 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.032640    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" event={"ID":"ebd442f7-6104-4efe-a863-647fea9cfd5a","Type":"ContainerDied","Data":"5dcf3b54e1626123c3192051c7055b07193be833856f3e6d00ee565d1ffd5fd2"}
May 28 08:26:36.032813 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.032693    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" event={"ID":"ebd442f7-6104-4efe-a863-647fea9cfd5a","Type":"ContainerDied","Data":"fe2a81314661eb640cde659e6fa0331b167626b7015172931a84c399e0219ccd"}
May 28 08:26:36.318940 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.318912    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:26:36.351945 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.351916    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ebd442f7-6104-4efe-a863-647fea9cfd5a-feast-data\") pod \"ebd442f7-6104-4efe-a863-647fea9cfd5a\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") "
May 28 08:26:36.352098 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.351994    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-offline-tls\") pod \"ebd442f7-6104-4efe-a863-647fea9cfd5a\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") "
May 28 08:26:36.352098 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.352045    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-online-tls\") pod \"ebd442f7-6104-4efe-a863-647fea9cfd5a\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") "
May 28 08:26:36.352098 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.352082    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-registry-tls\") pod \"ebd442f7-6104-4efe-a863-647fea9cfd5a\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") "
May 28 08:26:36.352258 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.352133    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-8tl6r\" (UniqueName: \"kubernetes.io/projected/ebd442f7-6104-4efe-a863-647fea9cfd5a-kube-api-access-8tl6r\") pod \"ebd442f7-6104-4efe-a863-647fea9cfd5a\" (UID: \"ebd442f7-6104-4efe-a863-647fea9cfd5a\") "
May 28 08:26:36.354993 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.354946    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "ebd442f7-6104-4efe-a863-647fea9cfd5a" (UID: "ebd442f7-6104-4efe-a863-647fea9cfd5a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:26:36.355141 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.355115    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/ebd442f7-6104-4efe-a863-647fea9cfd5a-kube-api-access-8tl6r" (OuterVolumeSpecName: "kube-api-access-8tl6r") pod "ebd442f7-6104-4efe-a863-647fea9cfd5a" (UID: "ebd442f7-6104-4efe-a863-647fea9cfd5a"). InnerVolumeSpecName "kube-api-access-8tl6r". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:26:36.355208 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.355153    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "ebd442f7-6104-4efe-a863-647fea9cfd5a" (UID: "ebd442f7-6104-4efe-a863-647fea9cfd5a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:26:36.355285 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.355264    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-online-tls" (OuterVolumeSpecName: "online-tls") pod "ebd442f7-6104-4efe-a863-647fea9cfd5a" (UID: "ebd442f7-6104-4efe-a863-647fea9cfd5a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:26:36.363423 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.363394    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/ebd442f7-6104-4efe-a863-647fea9cfd5a-feast-data" (OuterVolumeSpecName: "feast-data") pod "ebd442f7-6104-4efe-a863-647fea9cfd5a" (UID: "ebd442f7-6104-4efe-a863-647fea9cfd5a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:26:36.453597 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.453558    2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-online-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:26:36.453597 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.453591    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:26:36.453597 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.453602    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-8tl6r\" (UniqueName: \"kubernetes.io/projected/ebd442f7-6104-4efe-a863-647fea9cfd5a-kube-api-access-8tl6r\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:26:36.453849 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.453612    2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/ebd442f7-6104-4efe-a863-647fea9cfd5a-feast-data\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:26:36.453849 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:36.453621    2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/ebd442f7-6104-4efe-a863-647fea9cfd5a-offline-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:26:37.037724 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:37.037685    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs" event={"ID":"ebd442f7-6104-4efe-a863-647fea9cfd5a","Type":"ContainerDied","Data":"55d187d660e6fa5dd5e030d3c42d85258444f33691ebcb2bb0e4c0ecf47d86e7"}
May 28 08:26:37.037724 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:37.037695    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"
May 28 08:26:37.037724 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:37.037732    2577 scope.go:117] "RemoveContainer" containerID="5dcf3b54e1626123c3192051c7055b07193be833856f3e6d00ee565d1ffd5fd2"
May 28 08:26:37.046425 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:37.046400    2577 scope.go:117] "RemoveContainer" containerID="a6fa772cfada4315c85c621eb5ff6565d4f9af6738f590523a2cf63f5d2ddb69"
May 28 08:26:37.054492 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:37.054473    2577 scope.go:117] "RemoveContainer" containerID="fe2a81314661eb640cde659e6fa0331b167626b7015172931a84c399e0219ccd"
May 28 08:26:37.061735 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:37.061702    2577 scope.go:117] "RemoveContainer" containerID="b0f360d33dd44c3b4b87525a7659fa0ded0e9c82f3e1307c1333402c5c270483"
May 28 08:26:37.064818 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:37.064795    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"]
May 28 08:26:37.070456 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:37.070432    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-credit-scoring-76fbfc7469-nx8rs"]
May 28 08:26:37.070851 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:37.070827    2577 scope.go:117] "RemoveContainer" containerID="64a7e22f5080653808594df79a4ba7d6248011566569716859a3e53094918a01"
May 28 08:26:38.242457 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:26:38.242424    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" path="/var/lib/kubelet/pods/ebd442f7-6104-4efe-a863-647fea9cfd5a/volumes"
May 28 08:27:02.155932 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:27:02.155844    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:27:02.158967 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:27:02.158944    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:28:09.743244 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743206    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/redis-854544c579-v5k8d"]
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743708    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="feast-init"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743729    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="feast-init"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743755    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="registry"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743765    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="registry"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743826    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="feast-apply"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743836    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="feast-apply"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743849    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ad953d03-5213-4b68-8492-3c7037db2a76" containerName="postgres"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743857    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ad953d03-5213-4b68-8492-3c7037db2a76" containerName="postgres"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743873    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="offline"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743880    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="offline"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743892    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="online"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743901    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="online"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743969    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="offline"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743985    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="online"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.743995    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="ad953d03-5213-4b68-8492-3c7037db2a76" containerName="postgres"
May 28 08:28:09.745713 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.744006    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="ebd442f7-6104-4efe-a863-647fea9cfd5a" containerName="registry"
May 28 08:28:09.746759 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.746741    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-v5k8d"
May 28 08:28:09.749515 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.749490    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"kube-root-ca.crt\""
May 28 08:28:09.749515 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.749509    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"default-dockercfg-lp9p5\""
May 28 08:28:09.750683 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.750649    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"openshift-service-ca.crt\""
May 28 08:28:09.763607 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.763575    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-v5k8d"]
May 28 08:28:09.800301 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.800268    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4n4qp\" (UniqueName: \"kubernetes.io/projected/d248a6dd-0480-474a-90fa-6b98758564b5-kube-api-access-4n4qp\") pod \"redis-854544c579-v5k8d\" (UID: \"d248a6dd-0480-474a-90fa-6b98758564b5\") " pod="test-ns-feast-rest/redis-854544c579-v5k8d"
May 28 08:28:09.901722 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.901682    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4n4qp\" (UniqueName: \"kubernetes.io/projected/d248a6dd-0480-474a-90fa-6b98758564b5-kube-api-access-4n4qp\") pod \"redis-854544c579-v5k8d\" (UID: \"d248a6dd-0480-474a-90fa-6b98758564b5\") " pod="test-ns-feast-rest/redis-854544c579-v5k8d"
May 28 08:28:09.913433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:09.913397    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4n4qp\" (UniqueName: \"kubernetes.io/projected/d248a6dd-0480-474a-90fa-6b98758564b5-kube-api-access-4n4qp\") pod \"redis-854544c579-v5k8d\" (UID: \"d248a6dd-0480-474a-90fa-6b98758564b5\") " pod="test-ns-feast-rest/redis-854544c579-v5k8d"
May 28 08:28:10.056477 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:10.056430    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-v5k8d"
May 28 08:28:10.182207 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:10.182025    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/redis-854544c579-v5k8d"]
May 28 08:28:10.185070 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:28:10.185041    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd248a6dd_0480_474a_90fa_6b98758564b5.slice/crio-e2d9fc2d4856c919efe4e3a42a669efbc51a4714ce7f9a02214dd16d9ea86cc9 WatchSource:0}: Error finding container e2d9fc2d4856c919efe4e3a42a669efbc51a4714ce7f9a02214dd16d9ea86cc9: Status 404 returned error can't find the container with id e2d9fc2d4856c919efe4e3a42a669efbc51a4714ce7f9a02214dd16d9ea86cc9
May 28 08:28:10.314159 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:10.314061    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-v5k8d" event={"ID":"d248a6dd-0480-474a-90fa-6b98758564b5","Type":"ContainerStarted","Data":"e2d9fc2d4856c919efe4e3a42a669efbc51a4714ce7f9a02214dd16d9ea86cc9"}
May 28 08:28:12.320811 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:12.320769    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-v5k8d" event={"ID":"d248a6dd-0480-474a-90fa-6b98758564b5","Type":"ContainerStarted","Data":"2780fb118bbc7ec13f2158c5d02ac7a75f81bfaa8c8ac826a6e16e76756e064d"}
May 28 08:28:12.337823 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:12.337763    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/redis-854544c579-v5k8d" podStartSLOduration=1.9601312100000001 podStartE2EDuration="3.337743245s" podCreationTimestamp="2026-05-28 08:28:09 +0000 UTC" firstStartedPulling="2026-05-28 08:28:10.186789419 +0000 UTC m=+968.622968565" lastFinishedPulling="2026-05-28 08:28:11.564401439 +0000 UTC m=+970.000580600" observedRunningTime="2026-05-28 08:28:12.336738756 +0000 UTC m=+970.772917923" watchObservedRunningTime="2026-05-28 08:28:12.337743245 +0000 UTC m=+970.773922412"
May 28 08:28:15.164373 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:15.164335    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"]
May 28 08:28:15.167688 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:15.167647    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"
May 28 08:28:15.170601 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:15.170570    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"postgres-secret\""
May 28 08:28:15.175360 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:15.175328    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"]
May 28 08:28:15.248760 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:15.248719    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-dqnkd\" (UniqueName: \"kubernetes.io/projected/c7313cba-5129-460f-9fd7-9ed274e1abeb-kube-api-access-dqnkd\") pod \"postgres-7c7656cc6d-t9tdt\" (UID: \"c7313cba-5129-460f-9fd7-9ed274e1abeb\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"
May 28 08:28:15.248953 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:15.248776    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c7313cba-5129-460f-9fd7-9ed274e1abeb-postgresdata\") pod \"postgres-7c7656cc6d-t9tdt\" (UID: \"c7313cba-5129-460f-9fd7-9ed274e1abeb\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"
May 28 08:28:15.349569 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:15.349530    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-dqnkd\" (UniqueName: \"kubernetes.io/projected/c7313cba-5129-460f-9fd7-9ed274e1abeb-kube-api-access-dqnkd\") pod \"postgres-7c7656cc6d-t9tdt\" (UID: \"c7313cba-5129-460f-9fd7-9ed274e1abeb\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"
May 28 08:28:15.349784 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:15.349582    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c7313cba-5129-460f-9fd7-9ed274e1abeb-postgresdata\") pod \"postgres-7c7656cc6d-t9tdt\" (UID: \"c7313cba-5129-460f-9fd7-9ed274e1abeb\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"
May 28 08:28:15.349953 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:15.349937    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c7313cba-5129-460f-9fd7-9ed274e1abeb-postgresdata\") pod \"postgres-7c7656cc6d-t9tdt\" (UID: \"c7313cba-5129-460f-9fd7-9ed274e1abeb\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"
May 28 08:28:15.360352 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:15.360317    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-dqnkd\" (UniqueName: \"kubernetes.io/projected/c7313cba-5129-460f-9fd7-9ed274e1abeb-kube-api-access-dqnkd\") pod \"postgres-7c7656cc6d-t9tdt\" (UID: \"c7313cba-5129-460f-9fd7-9ed274e1abeb\") " pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"
May 28 08:28:15.480165 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:15.480081    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"
May 28 08:28:15.609908 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:15.609877    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"]
May 28 08:28:15.611893 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:28:15.611863    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7313cba_5129_460f_9fd7_9ed274e1abeb.slice/crio-56c22eafb04bc10f2a87911d4c66918890d57700f4b1950376ecf88e718c8f23 WatchSource:0}: Error finding container 56c22eafb04bc10f2a87911d4c66918890d57700f4b1950376ecf88e718c8f23: Status 404 returned error can't find the container with id 56c22eafb04bc10f2a87911d4c66918890d57700f4b1950376ecf88e718c8f23
May 28 08:28:16.334559 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:16.334524    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt" event={"ID":"c7313cba-5129-460f-9fd7-9ed274e1abeb","Type":"ContainerStarted","Data":"53fa49bc3d9b7868a16cb4b21795e3934a42f5272421e6c433fb1e2da6f880e7"}
May 28 08:28:16.334559 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:16.334562    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt" event={"ID":"c7313cba-5129-460f-9fd7-9ed274e1abeb","Type":"ContainerStarted","Data":"56c22eafb04bc10f2a87911d4c66918890d57700f4b1950376ecf88e718c8f23"}
May 28 08:28:16.350500 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:16.350449    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt" podStartSLOduration=1.350431206 podStartE2EDuration="1.350431206s" podCreationTimestamp="2026-05-28 08:28:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:28:16.349910061 +0000 UTC m=+974.786089228" watchObservedRunningTime="2026-05-28 08:28:16.350431206 +0000 UTC m=+974.786610372"
May 28 08:28:20.870860 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:20.870779    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"]
May 28 08:28:20.874699 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:20.874674    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:20.878582 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:20.878547    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-registry-tls\""
May 28 08:28:20.878756 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:20.878586    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-online-tls\""
May 28 08:28:20.878756 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:20.878591    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-dockercfg-wngnr\""
May 28 08:28:20.878756 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:20.878645    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-credit-scoring-intra-comm\""
May 28 08:28:20.886762 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:20.886740    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"]
May 28 08:28:20.998591 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:20.998549    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-online-tls\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:20.998591 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:20.998593    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/999874d8-2799-4945-b114-1109e3b83907-feast-data\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:20.999019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:20.998677    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-registry-tls\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:20.999019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:20.998745    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-s4lf7\" (UniqueName: \"kubernetes.io/projected/999874d8-2799-4945-b114-1109e3b83907-kube-api-access-s4lf7\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:21.100275 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:21.100233    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-registry-tls\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:21.100478 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:21.100303    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-s4lf7\" (UniqueName: \"kubernetes.io/projected/999874d8-2799-4945-b114-1109e3b83907-kube-api-access-s4lf7\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:21.100478 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:21.100356    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-online-tls\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:21.100478 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:21.100384    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/999874d8-2799-4945-b114-1109e3b83907-feast-data\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:21.100478 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:28:21.100460    2577 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-credit-scoring-online-tls: secret "feast-credit-scoring-online-tls" not found
May 28 08:28:21.100736 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:28:21.100529    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-online-tls podName:999874d8-2799-4945-b114-1109e3b83907 nodeName:}" failed. No retries permitted until 2026-05-28 08:28:21.600508836 +0000 UTC m=+980.036687980 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-online-tls") pod "feast-credit-scoring-8476f95c9-f7j7h" (UID: "999874d8-2799-4945-b114-1109e3b83907") : secret "feast-credit-scoring-online-tls" not found
May 28 08:28:21.100863 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:21.100843    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/999874d8-2799-4945-b114-1109e3b83907-feast-data\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:21.102890 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:21.102870    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-registry-tls\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:21.112629 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:21.112606    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-s4lf7\" (UniqueName: \"kubernetes.io/projected/999874d8-2799-4945-b114-1109e3b83907-kube-api-access-s4lf7\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:21.605386 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:21.605344    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-online-tls\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:21.607897 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:21.607874    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-online-tls\") pod \"feast-credit-scoring-8476f95c9-f7j7h\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") " pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:21.786645 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:21.786605    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:21.923495 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:21.923463    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"]
May 28 08:28:21.924966 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:28:21.924934    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod999874d8_2799_4945_b114_1109e3b83907.slice/crio-4176b697d5b1037c4f4ec74367b8b9e98fa992b3c607b97abdbe648fab3c7202 WatchSource:0}: Error finding container 4176b697d5b1037c4f4ec74367b8b9e98fa992b3c607b97abdbe648fab3c7202: Status 404 returned error can't find the container with id 4176b697d5b1037c4f4ec74367b8b9e98fa992b3c607b97abdbe648fab3c7202
May 28 08:28:22.351484 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:22.351429    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" event={"ID":"999874d8-2799-4945-b114-1109e3b83907","Type":"ContainerStarted","Data":"723a96e5487e5c2c12d8754851db5f953d6cf97f9028115633f69a5ba0c08cf8"}
May 28 08:28:22.351484 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:22.351475    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" event={"ID":"999874d8-2799-4945-b114-1109e3b83907","Type":"ContainerStarted","Data":"4176b697d5b1037c4f4ec74367b8b9e98fa992b3c607b97abdbe648fab3c7202"}
May 28 08:28:23.355241 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:23.355208    2577 generic.go:358] "Generic (PLEG): container finished" podID="999874d8-2799-4945-b114-1109e3b83907" containerID="723a96e5487e5c2c12d8754851db5f953d6cf97f9028115633f69a5ba0c08cf8" exitCode=0
May 28 08:28:23.355674 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:23.355302    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" event={"ID":"999874d8-2799-4945-b114-1109e3b83907","Type":"ContainerDied","Data":"723a96e5487e5c2c12d8754851db5f953d6cf97f9028115633f69a5ba0c08cf8"}
May 28 08:28:24.360076 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:24.360045    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" event={"ID":"999874d8-2799-4945-b114-1109e3b83907","Type":"ContainerStarted","Data":"b06062365a8dcf8ce0c253ff0387c33115d1c9ca32c6ef4e1352b038b237b9ae"}
May 28 08:28:31.382048 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:31.382018    2577 generic.go:358] "Generic (PLEG): container finished" podID="999874d8-2799-4945-b114-1109e3b83907" containerID="b06062365a8dcf8ce0c253ff0387c33115d1c9ca32c6ef4e1352b038b237b9ae" exitCode=0
May 28 08:28:31.382419 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:31.382077    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" event={"ID":"999874d8-2799-4945-b114-1109e3b83907","Type":"ContainerDied","Data":"b06062365a8dcf8ce0c253ff0387c33115d1c9ca32c6ef4e1352b038b237b9ae"}
May 28 08:28:32.387515 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:32.387470    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" event={"ID":"999874d8-2799-4945-b114-1109e3b83907","Type":"ContainerStarted","Data":"e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00"}
May 28 08:28:32.387515 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:32.387518    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" event={"ID":"999874d8-2799-4945-b114-1109e3b83907","Type":"ContainerStarted","Data":"be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c"}
May 28 08:28:32.417411 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:32.417361    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" podStartSLOduration=12.417340565 podStartE2EDuration="12.417340565s" podCreationTimestamp="2026-05-28 08:28:20 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:28:32.4154887 +0000 UTC m=+990.851667867" watchObservedRunningTime="2026-05-28 08:28:32.417340565 +0000 UTC m=+990.853519731"
May 28 08:28:33.787518 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:33.787468    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:33.787518 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:33.787523    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:33.789299 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:33.789266    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused"
May 28 08:28:33.789356 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:33.789300    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
May 28 08:28:36.787952 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:36.787896    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused"
May 28 08:28:36.788428 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:36.787911    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
May 28 08:28:39.787169 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:39.787122    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.31:6571: connect: connection refused"
May 28 08:28:39.792588 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:39.792560    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:39.793145 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:39.793123    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:39.796148 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:39.796123    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:41.786982 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:41.786936    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:42.787591 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:42.787556    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:43.427079 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:43.427045    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:28:46.609241 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.609197    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"]
May 28 08:28:46.614869 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.614847    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:46.618277 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.618252    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-online-tls\""
May 28 08:28:46.618416 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.618249    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-registry-tls\""
May 28 08:28:46.618416 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.618262    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-dockercfg-9px26\""
May 28 08:28:46.618855 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.618838    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast-rest\"/\"feast-driver-ranking-intra-comm\""
May 28 08:28:46.625081 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.625061    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"]
May 28 08:28:46.730178 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.730143    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-tq6hx\" (UniqueName: \"kubernetes.io/projected/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-kube-api-access-tq6hx\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:46.730379 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.730204    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-feast-data\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:46.730379 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.730314    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-registry-tls\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:46.730379 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.730361    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-online-tls\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:46.831044 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.831006    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-registry-tls\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:46.831215 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.831058    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-online-tls\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:46.831215 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.831102    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-tq6hx\" (UniqueName: \"kubernetes.io/projected/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-kube-api-access-tq6hx\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:46.831215 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:28:46.831141    2577 secret.go:189] Couldn't get secret test-ns-feast-rest/feast-driver-ranking-registry-tls: secret "feast-driver-ranking-registry-tls" not found
May 28 08:28:46.831215 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.831164    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-feast-data\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:46.831215 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:28:46.831214    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-registry-tls podName:2f39f1ed-ac84-4e21-9a52-cbfa6935917d nodeName:}" failed. No retries permitted until 2026-05-28 08:28:47.331197414 +0000 UTC m=+1005.767376557 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "registry-tls" (UniqueName: "kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-registry-tls") pod "feast-driver-ranking-778776cdd-x7xtx" (UID: "2f39f1ed-ac84-4e21-9a52-cbfa6935917d") : secret "feast-driver-ranking-registry-tls" not found
May 28 08:28:46.831641 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.831613    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-feast-data\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:46.833584 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.833565    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-online-tls\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:46.841075 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:46.841046    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-tq6hx\" (UniqueName: \"kubernetes.io/projected/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-kube-api-access-tq6hx\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:47.335495 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:47.335440    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-registry-tls\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:47.337953 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:47.337931    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-registry-tls\") pod \"feast-driver-ranking-778776cdd-x7xtx\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") " pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:47.525337 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:47.525296    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:28:47.655988 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:47.655946    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"]
May 28 08:28:47.659292 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:28:47.659264    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-burstable.slice/kubepods-burstable-pod2f39f1ed_ac84_4e21_9a52_cbfa6935917d.slice/crio-bd3f8796a4923fdf4d73c2d3b7d3e8c6c7dbba3c416fc90cef3d649c8eb46776 WatchSource:0}: Error finding container bd3f8796a4923fdf4d73c2d3b7d3e8c6c7dbba3c416fc90cef3d649c8eb46776: Status 404 returned error can't find the container with id bd3f8796a4923fdf4d73c2d3b7d3e8c6c7dbba3c416fc90cef3d649c8eb46776
May 28 08:28:48.443234 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:48.443186    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" event={"ID":"2f39f1ed-ac84-4e21-9a52-cbfa6935917d","Type":"ContainerStarted","Data":"7906234301f0b106597ca1beaf19347ae3e6df8118cb6f5e322c6124d81eaee5"}
May 28 08:28:48.443234 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:48.443229    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" event={"ID":"2f39f1ed-ac84-4e21-9a52-cbfa6935917d","Type":"ContainerStarted","Data":"bd3f8796a4923fdf4d73c2d3b7d3e8c6c7dbba3c416fc90cef3d649c8eb46776"}
May 28 08:28:52.456902 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:52.456869    2577 generic.go:358] "Generic (PLEG): container finished" podID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerID="7906234301f0b106597ca1beaf19347ae3e6df8118cb6f5e322c6124d81eaee5" exitCode=0
May 28 08:28:52.457291 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:52.456944    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" event={"ID":"2f39f1ed-ac84-4e21-9a52-cbfa6935917d","Type":"ContainerDied","Data":"7906234301f0b106597ca1beaf19347ae3e6df8118cb6f5e322c6124d81eaee5"}
May 28 08:28:53.464961 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:28:53.464923    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" event={"ID":"2f39f1ed-ac84-4e21-9a52-cbfa6935917d","Type":"ContainerStarted","Data":"33ee3a5a1a2ca6a1b2a9881742ac8e14657a6ccd36307ed28c5d6094ced3e866"}
May 28 08:29:00.487234 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:00.487200    2577 generic.go:358] "Generic (PLEG): container finished" podID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerID="33ee3a5a1a2ca6a1b2a9881742ac8e14657a6ccd36307ed28c5d6094ced3e866" exitCode=0
May 28 08:29:00.487610 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:00.487279    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" event={"ID":"2f39f1ed-ac84-4e21-9a52-cbfa6935917d","Type":"ContainerDied","Data":"33ee3a5a1a2ca6a1b2a9881742ac8e14657a6ccd36307ed28c5d6094ced3e866"}
May 28 08:29:01.493529 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:01.493491    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" event={"ID":"2f39f1ed-ac84-4e21-9a52-cbfa6935917d","Type":"ContainerStarted","Data":"95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9"}
May 28 08:29:01.493529 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:01.493532    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" event={"ID":"2f39f1ed-ac84-4e21-9a52-cbfa6935917d","Type":"ContainerStarted","Data":"281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf"}
May 28 08:29:01.517796 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:01.517739    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" podStartSLOduration=15.517718813 podStartE2EDuration="15.517718813s" podCreationTimestamp="2026-05-28 08:28:46 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:29:01.515725277 +0000 UTC m=+1019.951904443" watchObservedRunningTime="2026-05-28 08:29:01.517718813 +0000 UTC m=+1019.953897980"
May 28 08:29:02.526189 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:02.526142    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:29:02.526575 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:02.526199    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:29:02.527627 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:02.527600    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 28 08:29:02.527627 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:02.527610    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 28 08:29:05.525858 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:05.525812    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 28 08:29:05.526253 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:05.525823    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 28 08:29:07.525551 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:07.525502    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:29:07.525978 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:07.525564    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:29:08.526514 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:08.526464    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.32:6571: connect: connection refused"
May 28 08:29:08.531724 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:08.531698    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:29:08.534863 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:08.534837    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:29:11.526910 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:11.526870    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:29:11.527574 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:29:11.527554    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:30:26.140926 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:26.140837    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"]
May 28 08:30:26.141495 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:26.141236    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="registry" containerID="cri-o://be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c" gracePeriod=30
May 28 08:30:26.141495 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:26.141283    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="online" containerID="cri-o://e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00" gracePeriod=30
May 28 08:30:26.142859 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:26.142832    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"]
May 28 08:30:26.143123 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:26.143088    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="registry" containerID="cri-o://281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf" gracePeriod=30
May 28 08:30:26.143214 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:26.143146    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="online" containerID="cri-o://95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9" gracePeriod=30
May 28 08:30:27.768311 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:27.768280    2577 generic.go:358] "Generic (PLEG): container finished" podID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerID="95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9" exitCode=0
May 28 08:30:27.768768 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:27.768363    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" event={"ID":"2f39f1ed-ac84-4e21-9a52-cbfa6935917d","Type":"ContainerDied","Data":"95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9"}
May 28 08:30:27.770293 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:27.770270    2577 generic.go:358] "Generic (PLEG): container finished" podID="999874d8-2799-4945-b114-1109e3b83907" containerID="e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00" exitCode=0
May 28 08:30:27.770399 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:27.770329    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" event={"ID":"999874d8-2799-4945-b114-1109e3b83907","Type":"ContainerDied","Data":"e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00"}
May 28 08:30:28.533153 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.533109    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 28 08:30:28.561076 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.561043    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"]
May 28 08:30:28.561277 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.561239    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt" podUID="c7313cba-5129-460f-9fd7-9ed274e1abeb" containerName="postgres" containerID="cri-o://53fa49bc3d9b7868a16cb4b21795e3934a42f5272421e6c433fb1e2da6f880e7" gracePeriod=30
May 28 08:30:28.565901 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.565876    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-v5k8d"]
May 28 08:30:28.566092 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.566045    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast-rest/redis-854544c579-v5k8d" podUID="d248a6dd-0480-474a-90fa-6b98758564b5" containerName="redis" containerID="cri-o://2780fb118bbc7ec13f2158c5d02ac7a75f81bfaa8c8ac826a6e16e76756e064d" gracePeriod=30
May 28 08:30:28.775961 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.775901    2577 generic.go:358] "Generic (PLEG): container finished" podID="d248a6dd-0480-474a-90fa-6b98758564b5" containerID="2780fb118bbc7ec13f2158c5d02ac7a75f81bfaa8c8ac826a6e16e76756e064d" exitCode=0
May 28 08:30:28.776378 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.775957    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-v5k8d" event={"ID":"d248a6dd-0480-474a-90fa-6b98758564b5","Type":"ContainerDied","Data":"2780fb118bbc7ec13f2158c5d02ac7a75f81bfaa8c8ac826a6e16e76756e064d"}
May 28 08:30:28.778001 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.777975    2577 generic.go:358] "Generic (PLEG): container finished" podID="c7313cba-5129-460f-9fd7-9ed274e1abeb" containerID="53fa49bc3d9b7868a16cb4b21795e3934a42f5272421e6c433fb1e2da6f880e7" exitCode=0
May 28 08:30:28.778143 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.778025    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt" event={"ID":"c7313cba-5129-460f-9fd7-9ed274e1abeb","Type":"ContainerDied","Data":"53fa49bc3d9b7868a16cb4b21795e3934a42f5272421e6c433fb1e2da6f880e7"}
May 28 08:30:28.831365 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.831345    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-v5k8d"
May 28 08:30:28.834492 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.834474    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"
May 28 08:30:28.913640 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.913602    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4n4qp\" (UniqueName: \"kubernetes.io/projected/d248a6dd-0480-474a-90fa-6b98758564b5-kube-api-access-4n4qp\") pod \"d248a6dd-0480-474a-90fa-6b98758564b5\" (UID: \"d248a6dd-0480-474a-90fa-6b98758564b5\") "
May 28 08:30:28.913848 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.913684    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-dqnkd\" (UniqueName: \"kubernetes.io/projected/c7313cba-5129-460f-9fd7-9ed274e1abeb-kube-api-access-dqnkd\") pod \"c7313cba-5129-460f-9fd7-9ed274e1abeb\" (UID: \"c7313cba-5129-460f-9fd7-9ed274e1abeb\") "
May 28 08:30:28.913848 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.913716    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c7313cba-5129-460f-9fd7-9ed274e1abeb-postgresdata\") pod \"c7313cba-5129-460f-9fd7-9ed274e1abeb\" (UID: \"c7313cba-5129-460f-9fd7-9ed274e1abeb\") "
May 28 08:30:28.915994 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.915956    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7313cba-5129-460f-9fd7-9ed274e1abeb-kube-api-access-dqnkd" (OuterVolumeSpecName: "kube-api-access-dqnkd") pod "c7313cba-5129-460f-9fd7-9ed274e1abeb" (UID: "c7313cba-5129-460f-9fd7-9ed274e1abeb"). InnerVolumeSpecName "kube-api-access-dqnkd". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:30:28.915994 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.915984    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d248a6dd-0480-474a-90fa-6b98758564b5-kube-api-access-4n4qp" (OuterVolumeSpecName: "kube-api-access-4n4qp") pod "d248a6dd-0480-474a-90fa-6b98758564b5" (UID: "d248a6dd-0480-474a-90fa-6b98758564b5"). InnerVolumeSpecName "kube-api-access-4n4qp". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:30:28.943261 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:28.943217    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7313cba-5129-460f-9fd7-9ed274e1abeb-postgresdata" (OuterVolumeSpecName: "postgresdata") pod "c7313cba-5129-460f-9fd7-9ed274e1abeb" (UID: "c7313cba-5129-460f-9fd7-9ed274e1abeb"). InnerVolumeSpecName "postgresdata". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:30:29.014387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.014348    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4n4qp\" (UniqueName: \"kubernetes.io/projected/d248a6dd-0480-474a-90fa-6b98758564b5-kube-api-access-4n4qp\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:30:29.014387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.014380    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-dqnkd\" (UniqueName: \"kubernetes.io/projected/c7313cba-5129-460f-9fd7-9ed274e1abeb-kube-api-access-dqnkd\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:30:29.014387 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.014391    2577 reconciler_common.go:299] "Volume detached for volume \"postgresdata\" (UniqueName: \"kubernetes.io/empty-dir/c7313cba-5129-460f-9fd7-9ed274e1abeb-postgresdata\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:30:29.782494 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.782460    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/redis-854544c579-v5k8d"
May 28 08:30:29.782494 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.782469    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/redis-854544c579-v5k8d" event={"ID":"d248a6dd-0480-474a-90fa-6b98758564b5","Type":"ContainerDied","Data":"e2d9fc2d4856c919efe4e3a42a669efbc51a4714ce7f9a02214dd16d9ea86cc9"}
May 28 08:30:29.782996 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.782523    2577 scope.go:117] "RemoveContainer" containerID="2780fb118bbc7ec13f2158c5d02ac7a75f81bfaa8c8ac826a6e16e76756e064d"
May 28 08:30:29.783733 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.783710    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt" event={"ID":"c7313cba-5129-460f-9fd7-9ed274e1abeb","Type":"ContainerDied","Data":"56c22eafb04bc10f2a87911d4c66918890d57700f4b1950376ecf88e718c8f23"}
May 28 08:30:29.783783 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.783748    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"
May 28 08:30:29.791676 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.791634    2577 scope.go:117] "RemoveContainer" containerID="53fa49bc3d9b7868a16cb4b21795e3934a42f5272421e6c433fb1e2da6f880e7"
May 28 08:30:29.793695 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.793647    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
May 28 08:30:29.807634 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.807609    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/redis-854544c579-v5k8d"]
May 28 08:30:29.813917 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.813894    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/redis-854544c579-v5k8d"]
May 28 08:30:29.842116 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.842091    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"]
May 28 08:30:29.855186 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:29.855109    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/postgres-7c7656cc6d-t9tdt"]
May 28 08:30:30.241299 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:30.241217    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7313cba-5129-460f-9fd7-9ed274e1abeb" path="/var/lib/kubelet/pods/c7313cba-5129-460f-9fd7-9ed274e1abeb/volumes"
May 28 08:30:30.241533 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:30.241519    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d248a6dd-0480-474a-90fa-6b98758564b5" path="/var/lib/kubelet/pods/d248a6dd-0480-474a-90fa-6b98758564b5/volumes"
May 28 08:30:38.533262 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:38.533217    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 28 08:30:39.793701 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:39.793636    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
May 28 08:30:48.532592 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:48.532549    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="online" probeResult="failure" output="Get \"https://10.134.0.32:6567/health\": dial tcp 10.134.0.32:6567: connect: connection refused"
May 28 08:30:48.533019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:48.532718    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:30:49.793977 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:49.793920    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="online" probeResult="failure" output="Get \"https://10.134.0.31:6567/health\": dial tcp 10.134.0.31:6567: connect: connection refused"
May 28 08:30:49.794369 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:49.794086    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:30:56.823921 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.823895    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:30:56.826852 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.826832    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:30:56.869081 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.869045    2577 generic.go:358] "Generic (PLEG): container finished" podID="999874d8-2799-4945-b114-1109e3b83907" containerID="be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c" exitCode=137
May 28 08:30:56.869271 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.869128    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"
May 28 08:30:56.869271 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.869127    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" event={"ID":"999874d8-2799-4945-b114-1109e3b83907","Type":"ContainerDied","Data":"be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c"}
May 28 08:30:56.869271 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.869167    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h" event={"ID":"999874d8-2799-4945-b114-1109e3b83907","Type":"ContainerDied","Data":"4176b697d5b1037c4f4ec74367b8b9e98fa992b3c607b97abdbe648fab3c7202"}
May 28 08:30:56.869271 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.869193    2577 scope.go:117] "RemoveContainer" containerID="e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00"
May 28 08:30:56.871441 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.871413    2577 generic.go:358] "Generic (PLEG): container finished" podID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerID="281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf" exitCode=137
May 28 08:30:56.871579 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.871454    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" event={"ID":"2f39f1ed-ac84-4e21-9a52-cbfa6935917d","Type":"ContainerDied","Data":"281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf"}
May 28 08:30:56.871579 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.871493    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"
May 28 08:30:56.871804 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.871497    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx" event={"ID":"2f39f1ed-ac84-4e21-9a52-cbfa6935917d","Type":"ContainerDied","Data":"bd3f8796a4923fdf4d73c2d3b7d3e8c6c7dbba3c416fc90cef3d649c8eb46776"}
May 28 08:30:56.878299 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.878278    2577 scope.go:117] "RemoveContainer" containerID="be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c"
May 28 08:30:56.886675 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.886638    2577 scope.go:117] "RemoveContainer" containerID="b06062365a8dcf8ce0c253ff0387c33115d1c9ca32c6ef4e1352b038b237b9ae"
May 28 08:30:56.895792 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.895765    2577 scope.go:117] "RemoveContainer" containerID="723a96e5487e5c2c12d8754851db5f953d6cf97f9028115633f69a5ba0c08cf8"
May 28 08:30:56.903002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.902985    2577 scope.go:117] "RemoveContainer" containerID="e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00"
May 28 08:30:56.903274 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:30:56.903245    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00\": container with ID starting with e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00 not found: ID does not exist" containerID="e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00"
May 28 08:30:56.903353 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.903281    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00"} err="failed to get container status \"e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00\": rpc error: code = NotFound desc = could not find container \"e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00\": container with ID starting with e2d94147e7e9c3e3bee2ec17a8dd21283c38110984d27957e2300ee8c20ecd00 not found: ID does not exist"
May 28 08:30:56.903353 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.903301    2577 scope.go:117] "RemoveContainer" containerID="be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c"
May 28 08:30:56.903525 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:30:56.903507    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c\": container with ID starting with be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c not found: ID does not exist" containerID="be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c"
May 28 08:30:56.903571 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.903532    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c"} err="failed to get container status \"be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c\": rpc error: code = NotFound desc = could not find container \"be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c\": container with ID starting with be325e3b347d679b6078a4fb360fd313d96e3c572bc7e239c210bab7895fac9c not found: ID does not exist"
May 28 08:30:56.903571 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.903550    2577 scope.go:117] "RemoveContainer" containerID="b06062365a8dcf8ce0c253ff0387c33115d1c9ca32c6ef4e1352b038b237b9ae"
May 28 08:30:56.903776 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:30:56.903761    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b06062365a8dcf8ce0c253ff0387c33115d1c9ca32c6ef4e1352b038b237b9ae\": container with ID starting with b06062365a8dcf8ce0c253ff0387c33115d1c9ca32c6ef4e1352b038b237b9ae not found: ID does not exist" containerID="b06062365a8dcf8ce0c253ff0387c33115d1c9ca32c6ef4e1352b038b237b9ae"
May 28 08:30:56.903818 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.903780    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b06062365a8dcf8ce0c253ff0387c33115d1c9ca32c6ef4e1352b038b237b9ae"} err="failed to get container status \"b06062365a8dcf8ce0c253ff0387c33115d1c9ca32c6ef4e1352b038b237b9ae\": rpc error: code = NotFound desc = could not find container \"b06062365a8dcf8ce0c253ff0387c33115d1c9ca32c6ef4e1352b038b237b9ae\": container with ID starting with b06062365a8dcf8ce0c253ff0387c33115d1c9ca32c6ef4e1352b038b237b9ae not found: ID does not exist"
May 28 08:30:56.903818 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.903794    2577 scope.go:117] "RemoveContainer" containerID="723a96e5487e5c2c12d8754851db5f953d6cf97f9028115633f69a5ba0c08cf8"
May 28 08:30:56.903988 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:30:56.903970    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"723a96e5487e5c2c12d8754851db5f953d6cf97f9028115633f69a5ba0c08cf8\": container with ID starting with 723a96e5487e5c2c12d8754851db5f953d6cf97f9028115633f69a5ba0c08cf8 not found: ID does not exist" containerID="723a96e5487e5c2c12d8754851db5f953d6cf97f9028115633f69a5ba0c08cf8"
May 28 08:30:56.904029 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.903993    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"723a96e5487e5c2c12d8754851db5f953d6cf97f9028115633f69a5ba0c08cf8"} err="failed to get container status \"723a96e5487e5c2c12d8754851db5f953d6cf97f9028115633f69a5ba0c08cf8\": rpc error: code = NotFound desc = could not find container \"723a96e5487e5c2c12d8754851db5f953d6cf97f9028115633f69a5ba0c08cf8\": container with ID starting with 723a96e5487e5c2c12d8754851db5f953d6cf97f9028115633f69a5ba0c08cf8 not found: ID does not exist"
May 28 08:30:56.904029 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.904006    2577 scope.go:117] "RemoveContainer" containerID="95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9"
May 28 08:30:56.911106 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.911086    2577 scope.go:117] "RemoveContainer" containerID="281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf"
May 28 08:30:56.918544 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.918521    2577 scope.go:117] "RemoveContainer" containerID="33ee3a5a1a2ca6a1b2a9881742ac8e14657a6ccd36307ed28c5d6094ced3e866"
May 28 08:30:56.925701 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.925679    2577 scope.go:117] "RemoveContainer" containerID="7906234301f0b106597ca1beaf19347ae3e6df8118cb6f5e322c6124d81eaee5"
May 28 08:30:56.933104 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.933085    2577 scope.go:117] "RemoveContainer" containerID="95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9"
May 28 08:30:56.933391 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:30:56.933372    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9\": container with ID starting with 95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9 not found: ID does not exist" containerID="95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9"
May 28 08:30:56.933431 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.933400    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9"} err="failed to get container status \"95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9\": rpc error: code = NotFound desc = could not find container \"95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9\": container with ID starting with 95939bb66ef3a50a83408bec6667592683cf984be964586a5529283e1b2c4ba9 not found: ID does not exist"
May 28 08:30:56.933431 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.933419    2577 scope.go:117] "RemoveContainer" containerID="281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf"
May 28 08:30:56.933670 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:30:56.933636    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf\": container with ID starting with 281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf not found: ID does not exist" containerID="281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf"
May 28 08:30:56.933728 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.933679    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf"} err="failed to get container status \"281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf\": rpc error: code = NotFound desc = could not find container \"281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf\": container with ID starting with 281148601cdfad38274e01ac115e04d82590c6fef2f8cb904c7ebc30feef6ddf not found: ID does not exist"
May 28 08:30:56.933728 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.933697    2577 scope.go:117] "RemoveContainer" containerID="33ee3a5a1a2ca6a1b2a9881742ac8e14657a6ccd36307ed28c5d6094ced3e866"
May 28 08:30:56.933954 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:30:56.933936    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"33ee3a5a1a2ca6a1b2a9881742ac8e14657a6ccd36307ed28c5d6094ced3e866\": container with ID starting with 33ee3a5a1a2ca6a1b2a9881742ac8e14657a6ccd36307ed28c5d6094ced3e866 not found: ID does not exist" containerID="33ee3a5a1a2ca6a1b2a9881742ac8e14657a6ccd36307ed28c5d6094ced3e866"
May 28 08:30:56.934007 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.933960    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"33ee3a5a1a2ca6a1b2a9881742ac8e14657a6ccd36307ed28c5d6094ced3e866"} err="failed to get container status \"33ee3a5a1a2ca6a1b2a9881742ac8e14657a6ccd36307ed28c5d6094ced3e866\": rpc error: code = NotFound desc = could not find container \"33ee3a5a1a2ca6a1b2a9881742ac8e14657a6ccd36307ed28c5d6094ced3e866\": container with ID starting with 33ee3a5a1a2ca6a1b2a9881742ac8e14657a6ccd36307ed28c5d6094ced3e866 not found: ID does not exist"
May 28 08:30:56.934007 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.933977    2577 scope.go:117] "RemoveContainer" containerID="7906234301f0b106597ca1beaf19347ae3e6df8118cb6f5e322c6124d81eaee5"
May 28 08:30:56.934236 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:30:56.934219    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"7906234301f0b106597ca1beaf19347ae3e6df8118cb6f5e322c6124d81eaee5\": container with ID starting with 7906234301f0b106597ca1beaf19347ae3e6df8118cb6f5e322c6124d81eaee5 not found: ID does not exist" containerID="7906234301f0b106597ca1beaf19347ae3e6df8118cb6f5e322c6124d81eaee5"
May 28 08:30:56.934287 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.934244    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"7906234301f0b106597ca1beaf19347ae3e6df8118cb6f5e322c6124d81eaee5"} err="failed to get container status \"7906234301f0b106597ca1beaf19347ae3e6df8118cb6f5e322c6124d81eaee5\": rpc error: code = NotFound desc = could not find container \"7906234301f0b106597ca1beaf19347ae3e6df8118cb6f5e322c6124d81eaee5\": container with ID starting with 7906234301f0b106597ca1beaf19347ae3e6df8118cb6f5e322c6124d81eaee5 not found: ID does not exist"
May 28 08:30:56.956743 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.956711    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-feast-data\") pod \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") "
May 28 08:30:56.956937 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.956754    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-registry-tls\") pod \"999874d8-2799-4945-b114-1109e3b83907\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") "
May 28 08:30:56.956937 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.956780    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-online-tls\") pod \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") "
May 28 08:30:56.956937 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.956833    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-tq6hx\" (UniqueName: \"kubernetes.io/projected/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-kube-api-access-tq6hx\") pod \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") "
May 28 08:30:56.956937 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.956863    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/999874d8-2799-4945-b114-1109e3b83907-feast-data\") pod \"999874d8-2799-4945-b114-1109e3b83907\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") "
May 28 08:30:56.956937 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.956888    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-online-tls\") pod \"999874d8-2799-4945-b114-1109e3b83907\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") "
May 28 08:30:56.957203 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.956946    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-s4lf7\" (UniqueName: \"kubernetes.io/projected/999874d8-2799-4945-b114-1109e3b83907-kube-api-access-s4lf7\") pod \"999874d8-2799-4945-b114-1109e3b83907\" (UID: \"999874d8-2799-4945-b114-1109e3b83907\") "
May 28 08:30:56.957203 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.956970    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-registry-tls\") pod \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\" (UID: \"2f39f1ed-ac84-4e21-9a52-cbfa6935917d\") "
May 28 08:30:56.957323 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.957258    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-feast-data" (OuterVolumeSpecName: "feast-data") pod "2f39f1ed-ac84-4e21-9a52-cbfa6935917d" (UID: "2f39f1ed-ac84-4e21-9a52-cbfa6935917d"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:30:56.960097 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.959998    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "999874d8-2799-4945-b114-1109e3b83907" (UID: "999874d8-2799-4945-b114-1109e3b83907"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:30:56.960097 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.960003    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-online-tls" (OuterVolumeSpecName: "online-tls") pod "2f39f1ed-ac84-4e21-9a52-cbfa6935917d" (UID: "2f39f1ed-ac84-4e21-9a52-cbfa6935917d"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:30:56.960097 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.960021    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-online-tls" (OuterVolumeSpecName: "online-tls") pod "999874d8-2799-4945-b114-1109e3b83907" (UID: "999874d8-2799-4945-b114-1109e3b83907"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:30:56.960097 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.960027    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "2f39f1ed-ac84-4e21-9a52-cbfa6935917d" (UID: "2f39f1ed-ac84-4e21-9a52-cbfa6935917d"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:30:56.960097 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.960024    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/999874d8-2799-4945-b114-1109e3b83907-kube-api-access-s4lf7" (OuterVolumeSpecName: "kube-api-access-s4lf7") pod "999874d8-2799-4945-b114-1109e3b83907" (UID: "999874d8-2799-4945-b114-1109e3b83907"). InnerVolumeSpecName "kube-api-access-s4lf7". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:30:56.960097 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.960042    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-kube-api-access-tq6hx" (OuterVolumeSpecName: "kube-api-access-tq6hx") pod "2f39f1ed-ac84-4e21-9a52-cbfa6935917d" (UID: "2f39f1ed-ac84-4e21-9a52-cbfa6935917d"). InnerVolumeSpecName "kube-api-access-tq6hx". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:30:56.970169 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:56.970138    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/999874d8-2799-4945-b114-1109e3b83907-feast-data" (OuterVolumeSpecName: "feast-data") pod "999874d8-2799-4945-b114-1109e3b83907" (UID: "999874d8-2799-4945-b114-1109e3b83907"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:30:57.058569 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:57.058507    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-s4lf7\" (UniqueName: \"kubernetes.io/projected/999874d8-2799-4945-b114-1109e3b83907-kube-api-access-s4lf7\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:30:57.058569 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:57.058560    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:30:57.058569 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:57.058570    2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-feast-data\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:30:57.058569 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:57.058580    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:30:57.058569 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:57.058589    2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-online-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:30:57.059087 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:57.058597    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-tq6hx\" (UniqueName: \"kubernetes.io/projected/2f39f1ed-ac84-4e21-9a52-cbfa6935917d-kube-api-access-tq6hx\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:30:57.059087 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:57.058605    2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/999874d8-2799-4945-b114-1109e3b83907-feast-data\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:30:57.059087 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:57.058614    2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/999874d8-2799-4945-b114-1109e3b83907-online-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:30:57.195087 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:57.195051    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"]
May 28 08:30:57.199835 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:57.199808    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-credit-scoring-8476f95c9-f7j7h"]
May 28 08:30:57.212952 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:57.212920    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"]
May 28 08:30:57.216747 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:57.216720    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast-rest/feast-driver-ranking-778776cdd-x7xtx"]
May 28 08:30:58.242211 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:58.242177    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" path="/var/lib/kubelet/pods/2f39f1ed-ac84-4e21-9a52-cbfa6935917d/volumes"
May 28 08:30:58.242686 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:58.242672    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="999874d8-2799-4945-b114-1109e3b83907" path="/var/lib/kubelet/pods/999874d8-2799-4945-b114-1109e3b83907/volumes"
May 28 08:30:59.143037 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143000    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"]
May 28 08:30:59.143334 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143321    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7313cba-5129-460f-9fd7-9ed274e1abeb" containerName="postgres"
May 28 08:30:59.143389 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143336    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7313cba-5129-460f-9fd7-9ed274e1abeb" containerName="postgres"
May 28 08:30:59.143389 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143348    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d248a6dd-0480-474a-90fa-6b98758564b5" containerName="redis"
May 28 08:30:59.143389 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143354    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d248a6dd-0480-474a-90fa-6b98758564b5" containerName="redis"
May 28 08:30:59.143389 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143359    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="registry"
May 28 08:30:59.143389 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143365    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="registry"
May 28 08:30:59.143389 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143381    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="feast-apply"
May 28 08:30:59.143389 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143388    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="feast-apply"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143406    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="online"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143412    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="online"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143419    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="registry"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143424    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="registry"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143431    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="feast-init"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143436    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="feast-init"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143441    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="online"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143446    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="online"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143467    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="feast-init"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143476    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="feast-init"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143485    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="feast-apply"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143490    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="feast-apply"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143595    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="registry"
May 28 08:30:59.143603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143604    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="c7313cba-5129-460f-9fd7-9ed274e1abeb" containerName="postgres"
May 28 08:30:59.144036 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143610    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="online"
May 28 08:30:59.144036 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143618    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="d248a6dd-0480-474a-90fa-6b98758564b5" containerName="redis"
May 28 08:30:59.144036 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143626    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="999874d8-2799-4945-b114-1109e3b83907" containerName="online"
May 28 08:30:59.144036 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.143633    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="2f39f1ed-ac84-4e21-9a52-cbfa6935917d" containerName="registry"
May 28 08:30:59.149410 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.149388    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.158820 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.158801    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 08:30:59.159068 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.159052    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 08:30:59.160132 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.160114    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 08:30:59.160206 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.160116    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 08:30:59.160206 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.160114    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 08:30:59.160316 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.160119    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 08:30:59.160316 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.160159    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-mmdj4\""
May 28 08:30:59.181466 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.181428    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"]
May 28 08:30:59.276990 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.276955    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/97878de1-5e06-415e-bf5d-7e67e7939a93-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.277386 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.277015    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.277386 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.277073    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.277386 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.277105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.277386 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.277176    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-2cgj5\" (UniqueName: \"kubernetes.io/projected/97878de1-5e06-415e-bf5d-7e67e7939a93-kube-api-access-2cgj5\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.277386 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.277218    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.378186 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.378141    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.378369 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.378196    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-2cgj5\" (UniqueName: \"kubernetes.io/projected/97878de1-5e06-415e-bf5d-7e67e7939a93-kube-api-access-2cgj5\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.378369 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.378230    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.378369 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.378251    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/97878de1-5e06-415e-bf5d-7e67e7939a93-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.378369 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.378297    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.378369 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.378323    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.378700 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:30:59.378390    2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 08:30:59.378700 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:30:59.378488    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-ui-tls podName:97878de1-5e06-415e-bf5d-7e67e7939a93 nodeName:}" failed. No retries permitted until 2026-05-28 08:30:59.878465156 +0000 UTC m=+1138.314644305 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-nt6zp" (UID: "97878de1-5e06-415e-bf5d-7e67e7939a93") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 08:30:59.378849 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.378768    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/97878de1-5e06-415e-bf5d-7e67e7939a93-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.380940 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.380914    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.381096 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.381075    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.381139 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.381075    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.387525 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.387505    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-2cgj5\" (UniqueName: \"kubernetes.io/projected/97878de1-5e06-415e-bf5d-7e67e7939a93-kube-api-access-2cgj5\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.882833 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.882798    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:30:59.885300 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:30:59.885276    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-nt6zp\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:00.060699 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:00.060638    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:00.195384 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:00.195346    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"]
May 28 08:31:00.195912 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:31:00.195884    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod97878de1_5e06_415e_bf5d_7e67e7939a93.slice/crio-b3d1f958fa2ab6d13cd5984cc16f02a92fa3ee68299f24f50be4026170989de2 WatchSource:0}: Error finding container b3d1f958fa2ab6d13cd5984cc16f02a92fa3ee68299f24f50be4026170989de2: Status 404 returned error can't find the container with id b3d1f958fa2ab6d13cd5984cc16f02a92fa3ee68299f24f50be4026170989de2
May 28 08:31:00.197503 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:00.197486    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 08:31:00.887887 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:00.887847    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerStarted","Data":"b3d1f958fa2ab6d13cd5984cc16f02a92fa3ee68299f24f50be4026170989de2"}
May 28 08:31:17.943566 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:17.943523    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerStarted","Data":"510837d37f782c339873b8175af70e5717cc16d4d70b2d4299db7595c575a83b"}
May 28 08:31:21.956499 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:21.956459    2577 generic.go:358] "Generic (PLEG): container finished" podID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerID="510837d37f782c339873b8175af70e5717cc16d4d70b2d4299db7595c575a83b" exitCode=0
May 28 08:31:21.956934 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:21.956535    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerDied","Data":"510837d37f782c339873b8175af70e5717cc16d4d70b2d4299db7595c575a83b"}
May 28 08:31:22.960768 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:22.960731    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerStarted","Data":"1674ea6851b1aa4d4a54df22a838130032e5258d91647799bede468ac6d8c88d"}
May 28 08:31:27.976724 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:27.976622    2577 generic.go:358] "Generic (PLEG): container finished" podID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerID="1674ea6851b1aa4d4a54df22a838130032e5258d91647799bede468ac6d8c88d" exitCode=0
May 28 08:31:27.976724 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:27.976691    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerDied","Data":"1674ea6851b1aa4d4a54df22a838130032e5258d91647799bede468ac6d8c88d"}
May 28 08:31:28.984695 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:28.984627    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerStarted","Data":"146ecb2b46de81e396e552e297ff7df6b21a59bc763f6bbec0a794274a17d78a"}
May 28 08:31:28.985220 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:28.984707    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerStarted","Data":"6a9faf8fd093ad36487e3e92851c43b9e57cdef756da5a004a8b138ef736d683"}
May 28 08:31:28.985220 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:28.984727    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerStarted","Data":"a988681896e24ce81acaabc297758a48dd1ccaa77e32ce8d2a58c13b6fed3aec"}
May 28 08:31:28.985220 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:28.984741    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerStarted","Data":"17273b8498f86ed50a7e5da5efaffb932bcb36ae57a6119b037c3492ec409577"}
May 28 08:31:29.012496 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:29.012432    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podStartSLOduration=13.089067766 podStartE2EDuration="30.01241052s" podCreationTimestamp="2026-05-28 08:30:59 +0000 UTC" firstStartedPulling="2026-05-28 08:31:00.197615825 +0000 UTC m=+1138.633794972" lastFinishedPulling="2026-05-28 08:31:17.120958579 +0000 UTC m=+1155.557137726" observedRunningTime="2026-05-28 08:31:29.009339996 +0000 UTC m=+1167.445519173" watchObservedRunningTime="2026-05-28 08:31:29.01241052 +0000 UTC m=+1167.448589687"
May 28 08:31:30.060885 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:30.060849    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:30.061440 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:30.061409    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:30.061440 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:30.061445    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:30.061632 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:30.061459    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:30.061632 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:30.061472    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:30.061632 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:30.061483    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:30.061632 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:30.061495    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:30.061632 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:30.061510    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:30.063151 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:30.063118    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused"
May 28 08:31:30.063374 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:30.063133    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused"
May 28 08:31:30.063466 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:30.063185    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.33:8816: connect: connection refused"
May 28 08:31:30.063466 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:30.063226    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 28 08:31:33.061057 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:33.061000    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused"
May 28 08:31:33.061057 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:33.061039    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.33:8816: connect: connection refused"
May 28 08:31:33.061588 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:33.061051    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 28 08:31:33.061588 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:33.061185    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.33:6571: connect: connection refused"
May 28 08:31:36.062250 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:36.062211    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:36.062743 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:36.062413    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:36.062743 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:36.062683    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:36.063310 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:36.063285    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:36.063492 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:36.063461    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:36.063585 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:36.063556    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:36.067358 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:36.067334    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:31:37.015117 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:31:37.015089    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:32:02.178881 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:02.178849    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:32:02.183311 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:02.183276    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:32:38.965386 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:38.965344    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"]
May 28 08:32:38.965996 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:38.965821    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="registry" containerID="cri-o://17273b8498f86ed50a7e5da5efaffb932bcb36ae57a6119b037c3492ec409577" gracePeriod=30
May 28 08:32:38.965996 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:38.965945    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="online" containerID="cri-o://a988681896e24ce81acaabc297758a48dd1ccaa77e32ce8d2a58c13b6fed3aec" gracePeriod=30
May 28 08:32:38.966132 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:38.965997    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="ui" containerID="cri-o://146ecb2b46de81e396e552e297ff7df6b21a59bc763f6bbec0a794274a17d78a" gracePeriod=30
May 28 08:32:38.966132 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:38.965956    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="offline" containerID="cri-o://6a9faf8fd093ad36487e3e92851c43b9e57cdef756da5a004a8b138ef736d683" gracePeriod=30
May 28 08:32:39.388007 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.387969    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"]
May 28 08:32:39.392712 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.392688    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.400556 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.400534    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-xzk6k\""
May 28 08:32:39.404067 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.404040    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"]
May 28 08:32:39.520048 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.520013    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.520238 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.520081    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d208be6f-123c-4eb0-965a-fb901bcfc11a-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.520238 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.520110    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.520238 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.520126    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-ncrgh\" (UniqueName: \"kubernetes.io/projected/d208be6f-123c-4eb0-965a-fb901bcfc11a-kube-api-access-ncrgh\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.520238 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.520151    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.520238 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.520226    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.621348 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.621306    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.621550 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.621387    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d208be6f-123c-4eb0-965a-fb901bcfc11a-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.621550 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.621423    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.621550 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.621447    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-ncrgh\" (UniqueName: \"kubernetes.io/projected/d208be6f-123c-4eb0-965a-fb901bcfc11a-kube-api-access-ncrgh\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.621550 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.621482    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.621550 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.621516    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.621962 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.621904    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d208be6f-123c-4eb0-965a-fb901bcfc11a-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.624055 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.624022    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.624433 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.624409    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.624520 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.624484    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.624624 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.624601    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.636589 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.636562    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-ncrgh\" (UniqueName: \"kubernetes.io/projected/d208be6f-123c-4eb0-965a-fb901bcfc11a-kube-api-access-ncrgh\") pod \"feast-simple-feast-setup-664fcfcc58-zww47\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.702497 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.702460    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:39.835045 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:39.835000    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"]
May 28 08:32:39.837905 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:32:39.837877    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podd208be6f_123c_4eb0_965a_fb901bcfc11a.slice/crio-34c8ac63fb2f91177d1643de78763cc84b3a23c7e622acb5ce574f2147756dbe WatchSource:0}: Error finding container 34c8ac63fb2f91177d1643de78763cc84b3a23c7e622acb5ce574f2147756dbe: Status 404 returned error can't find the container with id 34c8ac63fb2f91177d1643de78763cc84b3a23c7e622acb5ce574f2147756dbe
May 28 08:32:40.199020 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:40.198984    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerStarted","Data":"cd27f40c799d87777672602cb1a0461229d4a46c38528495072994776ed8c1e7"}
May 28 08:32:40.199020 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:40.199019    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerStarted","Data":"34c8ac63fb2f91177d1643de78763cc84b3a23c7e622acb5ce574f2147756dbe"}
May 28 08:32:40.201727 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:40.201705    2577 generic.go:358] "Generic (PLEG): container finished" podID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerID="146ecb2b46de81e396e552e297ff7df6b21a59bc763f6bbec0a794274a17d78a" exitCode=0
May 28 08:32:40.201727 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:40.201727    2577 generic.go:358] "Generic (PLEG): container finished" podID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerID="a988681896e24ce81acaabc297758a48dd1ccaa77e32ce8d2a58c13b6fed3aec" exitCode=0
May 28 08:32:40.201873 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:40.201786    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerDied","Data":"146ecb2b46de81e396e552e297ff7df6b21a59bc763f6bbec0a794274a17d78a"}
May 28 08:32:40.201873 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:40.201825    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerDied","Data":"a988681896e24ce81acaabc297758a48dd1ccaa77e32ce8d2a58c13b6fed3aec"}
May 28 08:32:45.219726 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:45.219694    2577 generic.go:358] "Generic (PLEG): container finished" podID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerID="cd27f40c799d87777672602cb1a0461229d4a46c38528495072994776ed8c1e7" exitCode=0
May 28 08:32:45.220148 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:45.219770    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerDied","Data":"cd27f40c799d87777672602cb1a0461229d4a46c38528495072994776ed8c1e7"}
May 28 08:32:46.063331 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:46.063284    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused"
May 28 08:32:46.225792 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:46.225756    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerStarted","Data":"1474990d3fbc2a948cbdf152f3529b8fb341a8d3200660815442a6db73a1b22b"}
May 28 08:32:47.012635 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:47.012593    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 28 08:32:51.242504 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:51.242465    2577 generic.go:358] "Generic (PLEG): container finished" podID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerID="1474990d3fbc2a948cbdf152f3529b8fb341a8d3200660815442a6db73a1b22b" exitCode=0
May 28 08:32:51.242901 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:51.242541    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerDied","Data":"1474990d3fbc2a948cbdf152f3529b8fb341a8d3200660815442a6db73a1b22b"}
May 28 08:32:52.250126 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:52.250086    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerStarted","Data":"fe979990922dc78ea2916abad14ce87128682f627f69cdcde26d21bf07462f16"}
May 28 08:32:52.250610 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:52.250139    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerStarted","Data":"9af6ba155832d79d88c6e8fdc40cb566b5d9728622a6888871aa001e2a4362ee"}
May 28 08:32:52.250610 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:52.250157    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerStarted","Data":"d9b041c9817f0dd91c33d66f05a469121e9252fca8e89f7b49905839974b4989"}
May 28 08:32:52.250610 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:52.250170    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerStarted","Data":"179a6ad38ffff4626896a151bb8f215a2a070b9d8b49737dfd85b9cf32780d8a"}
May 28 08:32:52.278741 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:52.278677    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podStartSLOduration=13.278635566 podStartE2EDuration="13.278635566s" podCreationTimestamp="2026-05-28 08:32:39 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:32:52.276433817 +0000 UTC m=+1250.712613003" watchObservedRunningTime="2026-05-28 08:32:52.278635566 +0000 UTC m=+1250.714814734"
May 28 08:32:54.703786 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:54.702794    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:54.703786 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:54.702848    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:54.703786 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:54.702862    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:54.703786 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:54.702873    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:54.704675 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:54.704610    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.34:8816: connect: connection refused"
May 28 08:32:54.704825 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:54.704680    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 28 08:32:54.704825 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:54.704715    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 28 08:32:54.704825 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:54.704618    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.34:6571: connect: connection refused"
May 28 08:32:56.063699 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:56.063631    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused"
May 28 08:32:57.012522 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:57.012481    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 28 08:32:57.704450 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:57.704406    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:57.704862 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:57.704738    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:57.705113 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:57.705090    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:57.705198 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:57.705119    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:57.705198 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:57.705153    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:57.708996 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:57.708976    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:58.273718 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:58.273685    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:58.273718 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:58.273723    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:58.273957 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:58.273732    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:58.274386 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:58.274367    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:58.276815 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:58.276799    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:32:59.276828 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:32:59.276802    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:33:06.063736 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:06.063689    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.33:8443: connect: connection refused"
May 28 08:33:06.064174 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:06.063825    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:33:07.012803 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:07.012754    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="online" probeResult="failure" output="Get \"https://10.134.0.33:6567/health\": dial tcp 10.134.0.33:6567: connect: connection refused"
May 28 08:33:07.012987 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:07.012902    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:33:09.311476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.311441    2577 generic.go:358] "Generic (PLEG): container finished" podID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerID="6a9faf8fd093ad36487e3e92851c43b9e57cdef756da5a004a8b138ef736d683" exitCode=137
May 28 08:33:09.311476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.311472    2577 generic.go:358] "Generic (PLEG): container finished" podID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerID="17273b8498f86ed50a7e5da5efaffb932bcb36ae57a6119b037c3492ec409577" exitCode=137
May 28 08:33:09.311921 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.311510    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerDied","Data":"6a9faf8fd093ad36487e3e92851c43b9e57cdef756da5a004a8b138ef736d683"}
May 28 08:33:09.311921 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.311545    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerDied","Data":"17273b8498f86ed50a7e5da5efaffb932bcb36ae57a6119b037c3492ec409577"}
May 28 08:33:09.625696 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.625649    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:33:09.800239 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.800203    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/97878de1-5e06-415e-bf5d-7e67e7939a93-feast-data\") pod \"97878de1-5e06-415e-bf5d-7e67e7939a93\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") "
May 28 08:33:09.802535 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.800265    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-2cgj5\" (UniqueName: \"kubernetes.io/projected/97878de1-5e06-415e-bf5d-7e67e7939a93-kube-api-access-2cgj5\") pod \"97878de1-5e06-415e-bf5d-7e67e7939a93\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") "
May 28 08:33:09.802535 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.800297    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-offline-tls\") pod \"97878de1-5e06-415e-bf5d-7e67e7939a93\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") "
May 28 08:33:09.802535 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.800324    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-registry-tls\") pod \"97878de1-5e06-415e-bf5d-7e67e7939a93\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") "
May 28 08:33:09.802535 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.800366    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-online-tls\") pod \"97878de1-5e06-415e-bf5d-7e67e7939a93\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") "
May 28 08:33:09.802535 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.800534    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-ui-tls\") pod \"97878de1-5e06-415e-bf5d-7e67e7939a93\" (UID: \"97878de1-5e06-415e-bf5d-7e67e7939a93\") "
May 28 08:33:09.802535 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.800812    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/97878de1-5e06-415e-bf5d-7e67e7939a93-feast-data" (OuterVolumeSpecName: "feast-data") pod "97878de1-5e06-415e-bf5d-7e67e7939a93" (UID: "97878de1-5e06-415e-bf5d-7e67e7939a93"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:33:09.802535 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.800918    2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/97878de1-5e06-415e-bf5d-7e67e7939a93-feast-data\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:33:09.802931 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.802761    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "97878de1-5e06-415e-bf5d-7e67e7939a93" (UID: "97878de1-5e06-415e-bf5d-7e67e7939a93"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:33:09.802931 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.802839    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "97878de1-5e06-415e-bf5d-7e67e7939a93" (UID: "97878de1-5e06-415e-bf5d-7e67e7939a93"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:33:09.802931 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.802854    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "97878de1-5e06-415e-bf5d-7e67e7939a93" (UID: "97878de1-5e06-415e-bf5d-7e67e7939a93"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:33:09.803093 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.803049    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-online-tls" (OuterVolumeSpecName: "online-tls") pod "97878de1-5e06-415e-bf5d-7e67e7939a93" (UID: "97878de1-5e06-415e-bf5d-7e67e7939a93"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:33:09.803149 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.803105    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/97878de1-5e06-415e-bf5d-7e67e7939a93-kube-api-access-2cgj5" (OuterVolumeSpecName: "kube-api-access-2cgj5") pod "97878de1-5e06-415e-bf5d-7e67e7939a93" (UID: "97878de1-5e06-415e-bf5d-7e67e7939a93"). InnerVolumeSpecName "kube-api-access-2cgj5". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:33:09.901598 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.901509    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-2cgj5\" (UniqueName: \"kubernetes.io/projected/97878de1-5e06-415e-bf5d-7e67e7939a93-kube-api-access-2cgj5\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:33:09.901598 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.901539    2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-offline-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:33:09.901598 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.901547    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:33:09.901598 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.901556    2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-online-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:33:09.901598 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:09.901564    2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/97878de1-5e06-415e-bf5d-7e67e7939a93-ui-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:33:10.318006 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:10.317972    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp" event={"ID":"97878de1-5e06-415e-bf5d-7e67e7939a93","Type":"ContainerDied","Data":"b3d1f958fa2ab6d13cd5984cc16f02a92fa3ee68299f24f50be4026170989de2"}
May 28 08:33:10.318006 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:10.318004    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"
May 28 08:33:10.318499 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:10.318020    2577 scope.go:117] "RemoveContainer" containerID="146ecb2b46de81e396e552e297ff7df6b21a59bc763f6bbec0a794274a17d78a"
May 28 08:33:10.326907 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:10.326880    2577 scope.go:117] "RemoveContainer" containerID="6a9faf8fd093ad36487e3e92851c43b9e57cdef756da5a004a8b138ef736d683"
May 28 08:33:10.334378 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:10.334359    2577 scope.go:117] "RemoveContainer" containerID="a988681896e24ce81acaabc297758a48dd1ccaa77e32ce8d2a58c13b6fed3aec"
May 28 08:33:10.338948 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:10.338922    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"]
May 28 08:33:10.342415 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:10.342384    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-nt6zp"]
May 28 08:33:10.342800 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:10.342782    2577 scope.go:117] "RemoveContainer" containerID="17273b8498f86ed50a7e5da5efaffb932bcb36ae57a6119b037c3492ec409577"
May 28 08:33:10.350212 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:10.350182    2577 scope.go:117] "RemoveContainer" containerID="1674ea6851b1aa4d4a54df22a838130032e5258d91647799bede468ac6d8c88d"
May 28 08:33:10.357393 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:10.357376    2577 scope.go:117] "RemoveContainer" containerID="510837d37f782c339873b8175af70e5717cc16d4d70b2d4299db7595c575a83b"
May 28 08:33:12.242184 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:33:12.242148    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" path="/var/lib/kubelet/pods/97878de1-5e06-415e-bf5d-7e67e7939a93/volumes"
May 28 08:34:01.113928 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.113885    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"]
May 28 08:34:01.114476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114325    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="online"
May 28 08:34:01.114476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114340    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="online"
May 28 08:34:01.114476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114358    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="feast-init"
May 28 08:34:01.114476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114366    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="feast-init"
May 28 08:34:01.114476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114376    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="registry"
May 28 08:34:01.114476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114384    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="registry"
May 28 08:34:01.114476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114398    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="offline"
May 28 08:34:01.114476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114406    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="offline"
May 28 08:34:01.114476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114427    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="ui"
May 28 08:34:01.114476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114435    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="ui"
May 28 08:34:01.114476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114451    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="feast-apply"
May 28 08:34:01.114476 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114459    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="feast-apply"
May 28 08:34:01.115109 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114541    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="offline"
May 28 08:34:01.115109 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114556    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="registry"
May 28 08:34:01.115109 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114566    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="online"
May 28 08:34:01.115109 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.114579    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="97878de1-5e06-415e-bf5d-7e67e7939a93" containerName="ui"
May 28 08:34:01.118205 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.118184    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.121011 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.120986    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 08:34:01.121140 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.121058    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 08:34:01.121140 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.121085    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-g266m\""
May 28 08:34:01.121267 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.121161    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 08:34:01.121415 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.121395    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 08:34:01.121524 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.121490    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 08:34:01.121585 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.121559    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 08:34:01.128762 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.128739    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"]
May 28 08:34:01.217153 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.217111    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.217350 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.217185    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.217350 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.217219    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-4jpd2\" (UniqueName: \"kubernetes.io/projected/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-kube-api-access-4jpd2\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.217350 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.217259    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.217491 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.217344    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.217491 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.217405    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.318737 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.318688    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.318926 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.318749    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-4jpd2\" (UniqueName: \"kubernetes.io/projected/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-kube-api-access-4jpd2\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.318926 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.318783    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.318926 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.318803    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.318926 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.318837    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.318926 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.318896    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.319194 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:34:01.318948    2577 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-ui-tls: secret "feast-simple-feast-remote-setup-ui-tls" not found
May 28 08:34:01.319194 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:34:01.319036    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-ui-tls podName:96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3 nodeName:}" failed. No retries permitted until 2026-05-28 08:34:01.819014034 +0000 UTC m=+1320.255193178 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-ui-tls") pod "feast-simple-feast-remote-setup-5cff94746b-tc7pt" (UID: "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3") : secret "feast-simple-feast-remote-setup-ui-tls" not found
May 28 08:34:01.319306 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.319256    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.319511 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.319486    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.321454 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.321432    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.321562 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.321543    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.329967 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.329940    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-4jpd2\" (UniqueName: \"kubernetes.io/projected/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-kube-api-access-4jpd2\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.822435 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.822390    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:01.824910 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:01.824880    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-tc7pt\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:02.028709 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:02.028635    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:02.167905 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:02.167623    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"]
May 28 08:34:02.474845 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:02.474753    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" event={"ID":"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3","Type":"ContainerStarted","Data":"e941ef71020d0b6e58d08c698a4915595c8e87ce459bc2d771bfb8779a799307"}
May 28 08:34:02.475054 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:02.475041    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" event={"ID":"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3","Type":"ContainerStarted","Data":"e9751b7d22a869989a3bec57965fc67853d1c281aa2971b599a0041061c5c66b"}
May 28 08:34:07.490865 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:07.490832    2577 generic.go:358] "Generic (PLEG): container finished" podID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerID="e941ef71020d0b6e58d08c698a4915595c8e87ce459bc2d771bfb8779a799307" exitCode=0
May 28 08:34:07.491258 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:07.490914    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" event={"ID":"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3","Type":"ContainerDied","Data":"e941ef71020d0b6e58d08c698a4915595c8e87ce459bc2d771bfb8779a799307"}
May 28 08:34:08.496123 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:08.496085    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" event={"ID":"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3","Type":"ContainerStarted","Data":"528f5446f1eca3392678addb70ecb33ab45d6b0de8aef9fbdc19cab995595bd5"}
May 28 08:34:13.512188 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:13.512152    2577 generic.go:358] "Generic (PLEG): container finished" podID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerID="528f5446f1eca3392678addb70ecb33ab45d6b0de8aef9fbdc19cab995595bd5" exitCode=0
May 28 08:34:13.512595 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:13.512211    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" event={"ID":"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3","Type":"ContainerDied","Data":"528f5446f1eca3392678addb70ecb33ab45d6b0de8aef9fbdc19cab995595bd5"}
May 28 08:34:14.518851 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:14.518810    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" event={"ID":"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3","Type":"ContainerStarted","Data":"896a4dedfac17663e0a1cbdf2991e6087c1de2c0e8ba55881764978dd7da63e8"}
May 28 08:34:14.519230 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:14.518862    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" event={"ID":"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3","Type":"ContainerStarted","Data":"9d138ebe426c9d4fe3a64f88fa3d2edd30eba6bfd51a2385e6d8185e518daf86"}
May 28 08:34:14.519230 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:14.518879    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" event={"ID":"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3","Type":"ContainerStarted","Data":"0afb325e5f2da7cbf4a6a82d6939bb34c72203e84ab508b99d52bf4525573cb7"}
May 28 08:34:14.546084 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:14.546020    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podStartSLOduration=13.546003517 podStartE2EDuration="13.546003517s" podCreationTimestamp="2026-05-28 08:34:01 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:34:14.542641945 +0000 UTC m=+1332.978821116" watchObservedRunningTime="2026-05-28 08:34:14.546003517 +0000 UTC m=+1332.982182684"
May 28 08:34:17.029380 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:17.029343    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:17.030057 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:17.029477    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:17.030057 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:17.029495    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:17.031392 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:17.031341    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 28 08:34:17.031392 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:17.031382    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 28 08:34:17.031594 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:17.031351    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.35:8816: connect: connection refused"
May 28 08:34:20.030393 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:20.030318    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:20.030825 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:20.030442    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:20.030825 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:20.030765    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:20.030825 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:20.030798    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:20.031127 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:20.031104    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:20.031262 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:20.031245    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:20.034298 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:20.034281    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:20.540363 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:20.540326    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:34:20.543329 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:34:20.543306    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:35:22.363163 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:22.363121    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"]
May 28 08:35:22.364446 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:22.364272    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="offline" containerID="cri-o://9d138ebe426c9d4fe3a64f88fa3d2edd30eba6bfd51a2385e6d8185e518daf86" gracePeriod=30
May 28 08:35:22.364446 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:22.364283    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="ui" containerID="cri-o://896a4dedfac17663e0a1cbdf2991e6087c1de2c0e8ba55881764978dd7da63e8" gracePeriod=30
May 28 08:35:22.365179 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:22.364737    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="online" containerID="cri-o://0afb325e5f2da7cbf4a6a82d6939bb34c72203e84ab508b99d52bf4525573cb7" gracePeriod=30
May 28 08:35:22.519901 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:22.519857    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"]
May 28 08:35:22.520354 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:22.520299    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="registry" containerID="cri-o://179a6ad38ffff4626896a151bb8f215a2a070b9d8b49737dfd85b9cf32780d8a" gracePeriod=30
May 28 08:35:22.520518 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:22.520420    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="online" containerID="cri-o://d9b041c9817f0dd91c33d66f05a469121e9252fca8e89f7b49905839974b4989" gracePeriod=30
May 28 08:35:22.520518 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:22.520420    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="ui" containerID="cri-o://fe979990922dc78ea2916abad14ce87128682f627f69cdcde26d21bf07462f16" gracePeriod=30
May 28 08:35:22.520748 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:22.520422    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="offline" containerID="cri-o://9af6ba155832d79d88c6e8fdc40cb566b5d9728622a6888871aa001e2a4362ee" gracePeriod=30
May 28 08:35:23.733149 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:23.733114    2577 generic.go:358] "Generic (PLEG): container finished" podID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerID="896a4dedfac17663e0a1cbdf2991e6087c1de2c0e8ba55881764978dd7da63e8" exitCode=0
May 28 08:35:23.733149 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:23.733138    2577 generic.go:358] "Generic (PLEG): container finished" podID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerID="0afb325e5f2da7cbf4a6a82d6939bb34c72203e84ab508b99d52bf4525573cb7" exitCode=0
May 28 08:35:23.733599 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:23.733181    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" event={"ID":"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3","Type":"ContainerDied","Data":"896a4dedfac17663e0a1cbdf2991e6087c1de2c0e8ba55881764978dd7da63e8"}
May 28 08:35:23.733599 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:23.733213    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" event={"ID":"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3","Type":"ContainerDied","Data":"0afb325e5f2da7cbf4a6a82d6939bb34c72203e84ab508b99d52bf4525573cb7"}
May 28 08:35:23.735605 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:23.735579    2577 generic.go:358] "Generic (PLEG): container finished" podID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerID="fe979990922dc78ea2916abad14ce87128682f627f69cdcde26d21bf07462f16" exitCode=0
May 28 08:35:23.735605 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:23.735600    2577 generic.go:358] "Generic (PLEG): container finished" podID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerID="d9b041c9817f0dd91c33d66f05a469121e9252fca8e89f7b49905839974b4989" exitCode=0
May 28 08:35:23.735780 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:23.735623    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerDied","Data":"fe979990922dc78ea2916abad14ce87128682f627f69cdcde26d21bf07462f16"}
May 28 08:35:23.735780 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:23.735641    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerDied","Data":"d9b041c9817f0dd91c33d66f05a469121e9252fca8e89f7b49905839974b4989"}
May 28 08:35:28.274180 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:28.274136    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 28 08:35:28.274681 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:28.274152    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 28 08:35:30.031423 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:30.031366    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 28 08:35:30.541212 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:30.541162    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 28 08:35:38.274839 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:38.274796    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 28 08:35:38.275217 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:38.274796    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 28 08:35:40.031256 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:40.031216    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 28 08:35:40.541350 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:40.541305    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 28 08:35:48.273929 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:48.273880    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.34:8443: connect: connection refused"
May 28 08:35:48.273929 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:48.273923    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="online" probeResult="failure" output="Get \"https://10.134.0.34:6567/health\": dial tcp 10.134.0.34:6567: connect: connection refused"
May 28 08:35:48.274377 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:48.274017    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:35:48.274377 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:48.274054    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:35:50.031313 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:50.031206    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.35:8443: connect: connection refused"
May 28 08:35:50.031805 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:50.031385    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:35:50.540642 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:50.540597    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="online" probeResult="failure" output="Get \"https://10.134.0.35:6567/health\": dial tcp 10.134.0.35:6567: connect: connection refused"
May 28 08:35:50.540851 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:50.540735    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:35:52.822314 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:52.822283    2577 generic.go:358] "Generic (PLEG): container finished" podID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerID="9af6ba155832d79d88c6e8fdc40cb566b5d9728622a6888871aa001e2a4362ee" exitCode=137
May 28 08:35:52.822314 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:52.822311    2577 generic.go:358] "Generic (PLEG): container finished" podID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerID="179a6ad38ffff4626896a151bb8f215a2a070b9d8b49737dfd85b9cf32780d8a" exitCode=137
May 28 08:35:52.822797 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:52.822361    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerDied","Data":"9af6ba155832d79d88c6e8fdc40cb566b5d9728622a6888871aa001e2a4362ee"}
May 28 08:35:52.822797 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:52.822395    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerDied","Data":"179a6ad38ffff4626896a151bb8f215a2a070b9d8b49737dfd85b9cf32780d8a"}
May 28 08:35:52.824642 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:52.824619    2577 generic.go:358] "Generic (PLEG): container finished" podID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerID="9d138ebe426c9d4fe3a64f88fa3d2edd30eba6bfd51a2385e6d8185e518daf86" exitCode=137
May 28 08:35:52.824759 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:52.824693    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" event={"ID":"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3","Type":"ContainerDied","Data":"9d138ebe426c9d4fe3a64f88fa3d2edd30eba6bfd51a2385e6d8185e518daf86"}
May 28 08:35:53.015293 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.015269    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:35:53.165370 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.165348    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:35:53.175505 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.175483    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-ui-tls\") pod \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") "
May 28 08:35:53.175640 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.175523    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-offline-tls\") pod \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") "
May 28 08:35:53.175640 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.175561    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-registry-tls\") pod \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") "
May 28 08:35:53.175790 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.175642    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-feast-data\") pod \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") "
May 28 08:35:53.175790 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.175696    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-online-tls\") pod \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") "
May 28 08:35:53.175790 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.175749    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-4jpd2\" (UniqueName: \"kubernetes.io/projected/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-kube-api-access-4jpd2\") pod \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\" (UID: \"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3\") "
May 28 08:35:53.176045 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.176018    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" (UID: "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:35:53.176175 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.176123    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-feast-data" (OuterVolumeSpecName: "feast-data") pod "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" (UID: "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:35:53.177882 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.177856    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" (UID: "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:35:53.177982 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.177929    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-kube-api-access-4jpd2" (OuterVolumeSpecName: "kube-api-access-4jpd2") pod "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" (UID: "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3"). InnerVolumeSpecName "kube-api-access-4jpd2". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:35:53.178207 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.178192    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" (UID: "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:35:53.178271 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.178236    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-online-tls" (OuterVolumeSpecName: "online-tls") pod "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" (UID: "96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:35:53.276723 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.276686    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-online-tls\") pod \"d208be6f-123c-4eb0-965a-fb901bcfc11a\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") "
May 28 08:35:53.276723 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.276729    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-offline-tls\") pod \"d208be6f-123c-4eb0-965a-fb901bcfc11a\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") "
May 28 08:35:53.276978 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.276799    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-registry-tls\") pod \"d208be6f-123c-4eb0-965a-fb901bcfc11a\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") "
May 28 08:35:53.276978 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.276823    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-ncrgh\" (UniqueName: \"kubernetes.io/projected/d208be6f-123c-4eb0-965a-fb901bcfc11a-kube-api-access-ncrgh\") pod \"d208be6f-123c-4eb0-965a-fb901bcfc11a\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") "
May 28 08:35:53.276978 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.276878    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-ui-tls\") pod \"d208be6f-123c-4eb0-965a-fb901bcfc11a\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") "
May 28 08:35:53.276978 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.276898    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d208be6f-123c-4eb0-965a-fb901bcfc11a-feast-data\") pod \"d208be6f-123c-4eb0-965a-fb901bcfc11a\" (UID: \"d208be6f-123c-4eb0-965a-fb901bcfc11a\") "
May 28 08:35:53.277180 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.277113    2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-offline-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:35:53.277180 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.277128    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:35:53.277180 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.277143    2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-feast-data\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:35:53.277180 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.277157    2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-online-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:35:53.277180 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.277170    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-4jpd2\" (UniqueName: \"kubernetes.io/projected/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-kube-api-access-4jpd2\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:35:53.277425 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.277184    2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3-ui-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:35:53.277818 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.277779    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/d208be6f-123c-4eb0-965a-fb901bcfc11a-feast-data" (OuterVolumeSpecName: "feast-data") pod "d208be6f-123c-4eb0-965a-fb901bcfc11a" (UID: "d208be6f-123c-4eb0-965a-fb901bcfc11a"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:35:53.279176 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.279148    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "d208be6f-123c-4eb0-965a-fb901bcfc11a" (UID: "d208be6f-123c-4eb0-965a-fb901bcfc11a"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:35:53.279295 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.279203    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-online-tls" (OuterVolumeSpecName: "online-tls") pod "d208be6f-123c-4eb0-965a-fb901bcfc11a" (UID: "d208be6f-123c-4eb0-965a-fb901bcfc11a"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:35:53.279378 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.279348    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/d208be6f-123c-4eb0-965a-fb901bcfc11a-kube-api-access-ncrgh" (OuterVolumeSpecName: "kube-api-access-ncrgh") pod "d208be6f-123c-4eb0-965a-fb901bcfc11a" (UID: "d208be6f-123c-4eb0-965a-fb901bcfc11a"). InnerVolumeSpecName "kube-api-access-ncrgh". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:35:53.279502 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.279483    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "d208be6f-123c-4eb0-965a-fb901bcfc11a" (UID: "d208be6f-123c-4eb0-965a-fb901bcfc11a"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:35:53.279502 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.279497    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "d208be6f-123c-4eb0-965a-fb901bcfc11a" (UID: "d208be6f-123c-4eb0-965a-fb901bcfc11a"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:35:53.378360 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.378332    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:35:53.378462 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.378367    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-ncrgh\" (UniqueName: \"kubernetes.io/projected/d208be6f-123c-4eb0-965a-fb901bcfc11a-kube-api-access-ncrgh\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:35:53.378462 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.378383    2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-ui-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:35:53.378462 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.378396    2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/d208be6f-123c-4eb0-965a-fb901bcfc11a-feast-data\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:35:53.378462 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.378411    2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-online-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:35:53.378462 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.378423    2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/d208be6f-123c-4eb0-965a-fb901bcfc11a-offline-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:35:53.829850 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.829814    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47" event={"ID":"d208be6f-123c-4eb0-965a-fb901bcfc11a","Type":"ContainerDied","Data":"34c8ac63fb2f91177d1643de78763cc84b3a23c7e622acb5ce574f2147756dbe"}
May 28 08:35:53.829850 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.829841    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"
May 28 08:35:53.830394 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.829870    2577 scope.go:117] "RemoveContainer" containerID="fe979990922dc78ea2916abad14ce87128682f627f69cdcde26d21bf07462f16"
May 28 08:35:53.832191 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.832168    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt" event={"ID":"96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3","Type":"ContainerDied","Data":"e9751b7d22a869989a3bec57965fc67853d1c281aa2971b599a0041061c5c66b"}
May 28 08:35:53.832296 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.832241    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"
May 28 08:35:53.838739 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.838715    2577 scope.go:117] "RemoveContainer" containerID="9af6ba155832d79d88c6e8fdc40cb566b5d9728622a6888871aa001e2a4362ee"
May 28 08:35:53.850638 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.850620    2577 scope.go:117] "RemoveContainer" containerID="d9b041c9817f0dd91c33d66f05a469121e9252fca8e89f7b49905839974b4989"
May 28 08:35:53.858804 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.858782    2577 scope.go:117] "RemoveContainer" containerID="179a6ad38ffff4626896a151bb8f215a2a070b9d8b49737dfd85b9cf32780d8a"
May 28 08:35:53.860595 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.860570    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"]
May 28 08:35:53.864020 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.863998    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-zww47"]
May 28 08:35:53.867342 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.867326    2577 scope.go:117] "RemoveContainer" containerID="1474990d3fbc2a948cbdf152f3529b8fb341a8d3200660815442a6db73a1b22b"
May 28 08:35:53.877019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.876996    2577 scope.go:117] "RemoveContainer" containerID="cd27f40c799d87777672602cb1a0461229d4a46c38528495072994776ed8c1e7"
May 28 08:35:53.881150 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.881125    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"]
May 28 08:35:53.884405 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.884377    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-tc7pt"]
May 28 08:35:53.886231 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.886216    2577 scope.go:117] "RemoveContainer" containerID="896a4dedfac17663e0a1cbdf2991e6087c1de2c0e8ba55881764978dd7da63e8"
May 28 08:35:53.893870 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.893852    2577 scope.go:117] "RemoveContainer" containerID="9d138ebe426c9d4fe3a64f88fa3d2edd30eba6bfd51a2385e6d8185e518daf86"
May 28 08:35:53.901495 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.901474    2577 scope.go:117] "RemoveContainer" containerID="0afb325e5f2da7cbf4a6a82d6939bb34c72203e84ab508b99d52bf4525573cb7"
May 28 08:35:53.908606 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.908589    2577 scope.go:117] "RemoveContainer" containerID="528f5446f1eca3392678addb70ecb33ab45d6b0de8aef9fbdc19cab995595bd5"
May 28 08:35:53.915819 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:53.915799    2577 scope.go:117] "RemoveContainer" containerID="e941ef71020d0b6e58d08c698a4915595c8e87ce459bc2d771bfb8779a799307"
May 28 08:35:54.241937 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:54.241860    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" path="/var/lib/kubelet/pods/96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3/volumes"
May 28 08:35:54.242385 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:35:54.242371    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" path="/var/lib/kubelet/pods/d208be6f-123c-4eb0-965a-fb901bcfc11a/volumes"
May 28 08:36:33.372628 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.372585    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"]
May 28 08:36:33.373098 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373044    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="feast-apply"
May 28 08:36:33.373098 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373060    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="feast-apply"
May 28 08:36:33.373098 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373080    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="feast-init"
May 28 08:36:33.373098 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373089    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="feast-init"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373101    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="registry"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373110    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="registry"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373121    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="offline"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373129    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="offline"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373137    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="feast-init"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373145    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="feast-init"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373159    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="online"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373167    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="online"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373183    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="ui"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373191    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="ui"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373200    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="ui"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373208    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="ui"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373216    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="feast-apply"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373224    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="feast-apply"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373232    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="offline"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373240    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="offline"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373254    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="online"
May 28 08:36:33.373260 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373262    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="online"
May 28 08:36:33.373863 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373328    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="online"
May 28 08:36:33.373863 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373342    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="offline"
May 28 08:36:33.373863 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373354    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="ui"
May 28 08:36:33.373863 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373363    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="offline"
May 28 08:36:33.373863 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373373    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="96f9a1b4-7dd8-4735-9916-7d4aaacbd6c3" containerName="ui"
May 28 08:36:33.373863 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373383    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="registry"
May 28 08:36:33.373863 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.373392    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="d208be6f-123c-4eb0-965a-fb901bcfc11a" containerName="online"
May 28 08:36:33.377289 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.377269    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.380222 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.380197    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-ui-tls\""
May 28 08:36:33.380349 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.380245    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-offline-tls\""
May 28 08:36:33.380550 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.380530    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-online-tls\""
May 28 08:36:33.380698 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.380588    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"kube-root-ca.crt\""
May 28 08:36:33.380698 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.380650    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-feast\"/\"openshift-service-ca.crt\""
May 28 08:36:33.380698 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.380648    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-92hhd\""
May 28 08:36:33.380879 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.380734    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-registry-tls\""
May 28 08:36:33.388127 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.388095    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"]
May 28 08:36:33.489220 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.489178    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.489220 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.489220    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.489445 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.489243    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-cktxv\" (UniqueName: \"kubernetes.io/projected/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-kube-api-access-cktxv\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.489445 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.489340    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.489445 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.489368    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.489550 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.489447    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.589892 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.589857    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.589892 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.589901    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.590160 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.589922    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.590160 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.589944    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-cktxv\" (UniqueName: \"kubernetes.io/projected/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-kube-api-access-cktxv\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.590160 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.589977    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.590160 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.590085    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.590500 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.590465    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.592590 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.592559    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.592782 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.592764    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.592824 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.592796    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.592824 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.592809    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.599217 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.599192    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-cktxv\" (UniqueName: \"kubernetes.io/projected/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-kube-api-access-cktxv\") pod \"feast-simple-feast-setup-664fcfcc58-66lkc\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.689429 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.689336    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:33.826359 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.826331    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"]
May 28 08:36:33.828424 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:36:33.828374    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-podc7ee7d6d_2b50_4618_91d8_a7f5acac7610.slice/crio-4d08c601c509e652f46e8ceb4611893bab3d745dec728568a1fc494a62b5f472 WatchSource:0}: Error finding container 4d08c601c509e652f46e8ceb4611893bab3d745dec728568a1fc494a62b5f472: Status 404 returned error can't find the container with id 4d08c601c509e652f46e8ceb4611893bab3d745dec728568a1fc494a62b5f472
May 28 08:36:33.830212 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.830195    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 08:36:33.947517 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.947436    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerStarted","Data":"93aff9731dabfb4a8bdb419d9da74c3c658b7086dbe232e617e0a2b467817a5e"}
May 28 08:36:33.947517 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:33.947476    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerStarted","Data":"4d08c601c509e652f46e8ceb4611893bab3d745dec728568a1fc494a62b5f472"}
May 28 08:36:38.964729 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:38.964624    2577 generic.go:358] "Generic (PLEG): container finished" podID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerID="93aff9731dabfb4a8bdb419d9da74c3c658b7086dbe232e617e0a2b467817a5e" exitCode=0
May 28 08:36:38.964729 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:38.964711    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerDied","Data":"93aff9731dabfb4a8bdb419d9da74c3c658b7086dbe232e617e0a2b467817a5e"}
May 28 08:36:39.969776 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:39.969742    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerStarted","Data":"cbb1f8b196e3ced685043991a8624abec1b43ea556b659743e83e4e7769220a5"}
May 28 08:36:44.989153 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:44.989060    2577 generic.go:358] "Generic (PLEG): container finished" podID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerID="cbb1f8b196e3ced685043991a8624abec1b43ea556b659743e83e4e7769220a5" exitCode=0
May 28 08:36:44.989153 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:44.989129    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerDied","Data":"cbb1f8b196e3ced685043991a8624abec1b43ea556b659743e83e4e7769220a5"}
May 28 08:36:45.996649 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:45.996608    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerStarted","Data":"8038e1e11474dca6ecdfb295e3fdb57ba820500260de2c5cb5ab41ff6f3ecb3a"}
May 28 08:36:45.997169 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:45.996678    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerStarted","Data":"f396e793f322043160802136c2ebfb2ffc2a58703703010b502708e0cfead5b5"}
May 28 08:36:45.997169 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:45.996695    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerStarted","Data":"7e3bdc307ae073cd276f816580bfdc3199aa6b502e8329cf95d71c98011c2fd7"}
May 28 08:36:45.997169 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:45.996708    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerStarted","Data":"5a0dbb008a7ab668699d517e37110d5c978fbcbce8ab091cc6b39171231f3b4d"}
May 28 08:36:46.026107 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:46.026046    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podStartSLOduration=13.0260248 podStartE2EDuration="13.0260248s" podCreationTimestamp="2026-05-28 08:36:33 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:36:46.022822532 +0000 UTC m=+1484.459001710" watchObservedRunningTime="2026-05-28 08:36:46.0260248 +0000 UTC m=+1484.462203966"
May 28 08:36:48.690370 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:48.690227    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:48.690370 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:48.690277    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:48.690370 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:48.690291    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:48.690370 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:48.690302    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:48.692505 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:48.692468    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 28 08:36:48.692647 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:48.692507    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 28 08:36:48.692647 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:48.692529    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.36:8816: connect: connection refused"
May 28 08:36:48.692647 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:48.692595    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.36:6571: connect: connection refused"
May 28 08:36:51.691030 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:51.690972    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:51.691474 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:51.691197    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:51.691474 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:51.691292    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:51.691474 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:51.691321    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:51.691474 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:51.691473    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:51.691790 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:51.691770    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:51.691852 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:51.691820    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:51.695609 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:51.695594    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:52.018497 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:52.018468    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:52.018497 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:52.018503    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:52.018837 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:52.018817    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:36:52.021232 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:36:52.021214    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:37:02.202968 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:02.202931    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:37:02.208990 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:02.208968    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:37:53.961322 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:53.961280    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"]
May 28 08:37:53.962352 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:53.962295    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="registry" containerID="cri-o://5a0dbb008a7ab668699d517e37110d5c978fbcbce8ab091cc6b39171231f3b4d" gracePeriod=30
May 28 08:37:53.962567 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:53.962346    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="ui" containerID="cri-o://8038e1e11474dca6ecdfb295e3fdb57ba820500260de2c5cb5ab41ff6f3ecb3a" gracePeriod=30
May 28 08:37:53.962838 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:53.962373    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="offline" containerID="cri-o://f396e793f322043160802136c2ebfb2ffc2a58703703010b502708e0cfead5b5" gracePeriod=30
May 28 08:37:53.962937 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:53.962376    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="online" containerID="cri-o://7e3bdc307ae073cd276f816580bfdc3199aa6b502e8329cf95d71c98011c2fd7" gracePeriod=30
May 28 08:37:54.290086 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.290037    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"]
May 28 08:37:54.295243 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.295219    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.298376 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.298348    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-feast\"/\"feast-simple-feast-setup-dockercfg-2cm8n\""
May 28 08:37:54.306314 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.306063    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"]
May 28 08:37:54.381081 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.381036    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a4aff123-3b0a-4245-ad33-bd3923ebc30f-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.381308 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.381105    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.381308 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.381175    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-mpx4b\" (UniqueName: \"kubernetes.io/projected/a4aff123-3b0a-4245-ad33-bd3923ebc30f-kube-api-access-mpx4b\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.381308 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.381207    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.381308 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.381262    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.381308 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.381301    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.482534 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.482364    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.483146 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:37:54.482506    2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-online-tls: secret "feast-simple-feast-setup-online-tls" not found
May 28 08:37:54.483146 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:37:54.482893    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-online-tls podName:a4aff123-3b0a-4245-ad33-bd3923ebc30f nodeName:}" failed. No retries permitted until 2026-05-28 08:37:54.982865341 +0000 UTC m=+1553.419044503 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "online-tls" (UniqueName: "kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-online-tls") pod "feast-simple-feast-setup-664fcfcc58-lmhp9" (UID: "a4aff123-3b0a-4245-ad33-bd3923ebc30f") : secret "feast-simple-feast-setup-online-tls" not found
May 28 08:37:54.483342 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.483255    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a4aff123-3b0a-4245-ad33-bd3923ebc30f-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.483342 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.483320    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a4aff123-3b0a-4245-ad33-bd3923ebc30f-feast-data\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.483480 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.483389    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.483480 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.483431    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-mpx4b\" (UniqueName: \"kubernetes.io/projected/a4aff123-3b0a-4245-ad33-bd3923ebc30f-kube-api-access-mpx4b\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.483480 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.483468    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.483688 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.483536    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.483688 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:37:54.483675    2577 secret.go:189] Couldn't get secret test-ns-feast/feast-simple-feast-setup-ui-tls: secret "feast-simple-feast-setup-ui-tls" not found
May 28 08:37:54.483815 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:37:54.483733    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-ui-tls podName:a4aff123-3b0a-4245-ad33-bd3923ebc30f nodeName:}" failed. No retries permitted until 2026-05-28 08:37:54.983714138 +0000 UTC m=+1553.419893288 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "ui-tls" (UniqueName: "kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-ui-tls") pod "feast-simple-feast-setup-664fcfcc58-lmhp9" (UID: "a4aff123-3b0a-4245-ad33-bd3923ebc30f") : secret "feast-simple-feast-setup-ui-tls" not found
May 28 08:37:54.486468 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.486407    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-registry-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.486607 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.486584    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-offline-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.495695 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.495637    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-mpx4b\" (UniqueName: \"kubernetes.io/projected/a4aff123-3b0a-4245-ad33-bd3923ebc30f-kube-api-access-mpx4b\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.987569 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.987460    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.987569 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.987514    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.990009 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.989983    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-online-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:54.990116 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:54.990039    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-ui-tls\") pod \"feast-simple-feast-setup-664fcfcc58-lmhp9\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") " pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:55.206427 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:55.206394    2577 generic.go:358] "Generic (PLEG): container finished" podID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerID="8038e1e11474dca6ecdfb295e3fdb57ba820500260de2c5cb5ab41ff6f3ecb3a" exitCode=0
May 28 08:37:55.206427 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:55.206421    2577 generic.go:358] "Generic (PLEG): container finished" podID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerID="7e3bdc307ae073cd276f816580bfdc3199aa6b502e8329cf95d71c98011c2fd7" exitCode=0
May 28 08:37:55.206629 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:55.206472    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerDied","Data":"8038e1e11474dca6ecdfb295e3fdb57ba820500260de2c5cb5ab41ff6f3ecb3a"}
May 28 08:37:55.206629 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:55.206510    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerDied","Data":"7e3bdc307ae073cd276f816580bfdc3199aa6b502e8329cf95d71c98011c2fd7"}
May 28 08:37:55.209631 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:55.209611    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:37:55.348286 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:55.348261    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"]
May 28 08:37:55.349713 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:37:55.349685    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-poda4aff123_3b0a_4245_ad33_bd3923ebc30f.slice/crio-224ca5529722fc456349c2dc832e0eacaa9b8f197f90a1e994c45806d5930b8a WatchSource:0}: Error finding container 224ca5529722fc456349c2dc832e0eacaa9b8f197f90a1e994c45806d5930b8a: Status 404 returned error can't find the container with id 224ca5529722fc456349c2dc832e0eacaa9b8f197f90a1e994c45806d5930b8a
May 28 08:37:56.211196 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:56.211151    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerStarted","Data":"3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a"}
May 28 08:37:56.211196 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:37:56.211191    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerStarted","Data":"224ca5529722fc456349c2dc832e0eacaa9b8f197f90a1e994c45806d5930b8a"}
May 28 08:38:00.224318 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:00.224232    2577 generic.go:358] "Generic (PLEG): container finished" podID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerID="3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a" exitCode=0
May 28 08:38:00.224318 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:00.224286    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerDied","Data":"3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a"}
May 28 08:38:01.229155 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:01.229119    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerStarted","Data":"8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636"}
May 28 08:38:01.692412 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:01.692363    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 28 08:38:02.018932 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:02.018880    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 28 08:38:06.247078 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:06.246993    2577 generic.go:358] "Generic (PLEG): container finished" podID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerID="8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636" exitCode=0
May 28 08:38:06.247472 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:06.247068    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerDied","Data":"8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636"}
May 28 08:38:07.253928 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:07.253884    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerStarted","Data":"8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5"}
May 28 08:38:07.254407 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:07.253939    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerStarted","Data":"3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072"}
May 28 08:38:07.254407 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:07.253956    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerStarted","Data":"3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d"}
May 28 08:38:07.254407 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:07.253971    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerStarted","Data":"58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206"}
May 28 08:38:07.282869 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:07.282812    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podStartSLOduration=13.282790072 podStartE2EDuration="13.282790072s" podCreationTimestamp="2026-05-28 08:37:54 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:38:07.280266581 +0000 UTC m=+1565.716445772" watchObservedRunningTime="2026-05-28 08:38:07.282790072 +0000 UTC m=+1565.718969239"
May 28 08:38:10.209950 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:10.209910    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:10.212566 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:10.210445    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:10.212566 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:10.210469    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:10.212566 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:10.210483    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:10.212566 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:10.212233    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 28 08:38:10.212566 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:10.212256    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="registry" probeResult="failure" output="dial tcp 10.134.0.37:6571: connect: connection refused"
May 28 08:38:10.212566 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:10.212293    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 28 08:38:10.212566 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:10.212317    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.37:8816: connect: connection refused"
May 28 08:38:11.692217 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:11.692169    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 28 08:38:12.019183 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:12.019137    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 28 08:38:13.210989 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:13.210955    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:13.211467 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:13.211184    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:13.211467 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:13.211375    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:13.211467 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:13.211398    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:13.211625 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:13.211570    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:13.211854 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:13.211837    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:13.211900 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:13.211891    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:13.215603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:13.215587    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:13.277638 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:13.277566    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:13.277638 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:13.277606    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:13.278150 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:13.278135    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:13.280331 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:13.280312    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:38:21.691595 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:21.691538    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.36:8443: connect: connection refused"
May 28 08:38:21.692115 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:21.691746    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:38:22.019300 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:22.019259    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="online" probeResult="failure" output="Get \"https://10.134.0.36:6567/health\": dial tcp 10.134.0.36:6567: connect: connection refused"
May 28 08:38:22.019508 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:22.019397    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:38:24.313593 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.313557    2577 generic.go:358] "Generic (PLEG): container finished" podID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerID="f396e793f322043160802136c2ebfb2ffc2a58703703010b502708e0cfead5b5" exitCode=137
May 28 08:38:24.313593 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.313586    2577 generic.go:358] "Generic (PLEG): container finished" podID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerID="5a0dbb008a7ab668699d517e37110d5c978fbcbce8ab091cc6b39171231f3b4d" exitCode=137
May 28 08:38:24.314045 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.313635    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerDied","Data":"f396e793f322043160802136c2ebfb2ffc2a58703703010b502708e0cfead5b5"}
May 28 08:38:24.314045 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.313693    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerDied","Data":"5a0dbb008a7ab668699d517e37110d5c978fbcbce8ab091cc6b39171231f3b4d"}
May 28 08:38:24.617778 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.617754    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:38:24.751480 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.751434    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-registry-tls\") pod \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") "
May 28 08:38:24.751724 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.751503    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-online-tls\") pod \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") "
May 28 08:38:24.751724 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.751541    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-ui-tls\") pod \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") "
May 28 08:38:24.751724 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.751567    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-cktxv\" (UniqueName: \"kubernetes.io/projected/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-kube-api-access-cktxv\") pod \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") "
May 28 08:38:24.751724 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.751594    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-feast-data\") pod \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") "
May 28 08:38:24.751724 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.751647    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-offline-tls\") pod \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\" (UID: \"c7ee7d6d-2b50-4618-91d8-a7f5acac7610\") "
May 28 08:38:24.752235 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.752199    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-feast-data" (OuterVolumeSpecName: "feast-data") pod "c7ee7d6d-2b50-4618-91d8-a7f5acac7610" (UID: "c7ee7d6d-2b50-4618-91d8-a7f5acac7610"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:38:24.753961 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.753928    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-kube-api-access-cktxv" (OuterVolumeSpecName: "kube-api-access-cktxv") pod "c7ee7d6d-2b50-4618-91d8-a7f5acac7610" (UID: "c7ee7d6d-2b50-4618-91d8-a7f5acac7610"). InnerVolumeSpecName "kube-api-access-cktxv". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:38:24.753961 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.753948    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-online-tls" (OuterVolumeSpecName: "online-tls") pod "c7ee7d6d-2b50-4618-91d8-a7f5acac7610" (UID: "c7ee7d6d-2b50-4618-91d8-a7f5acac7610"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:38:24.754131 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.753973    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "c7ee7d6d-2b50-4618-91d8-a7f5acac7610" (UID: "c7ee7d6d-2b50-4618-91d8-a7f5acac7610"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:38:24.754203 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.754189    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "c7ee7d6d-2b50-4618-91d8-a7f5acac7610" (UID: "c7ee7d6d-2b50-4618-91d8-a7f5acac7610"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:38:24.754242 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.754222    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "c7ee7d6d-2b50-4618-91d8-a7f5acac7610" (UID: "c7ee7d6d-2b50-4618-91d8-a7f5acac7610"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:38:24.852831 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.852742    2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-offline-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:38:24.852831 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.852773    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:38:24.852831 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.852782    2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-online-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:38:24.852831 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.852792    2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-ui-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:38:24.852831 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.852802    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-cktxv\" (UniqueName: \"kubernetes.io/projected/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-kube-api-access-cktxv\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:38:24.852831 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:24.852812    2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/c7ee7d6d-2b50-4618-91d8-a7f5acac7610-feast-data\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:38:25.318967 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:25.318933    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc" event={"ID":"c7ee7d6d-2b50-4618-91d8-a7f5acac7610","Type":"ContainerDied","Data":"4d08c601c509e652f46e8ceb4611893bab3d745dec728568a1fc494a62b5f472"}
May 28 08:38:25.319406 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:25.318986    2577 scope.go:117] "RemoveContainer" containerID="8038e1e11474dca6ecdfb295e3fdb57ba820500260de2c5cb5ab41ff6f3ecb3a"
May 28 08:38:25.319406 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:25.318997    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"
May 28 08:38:25.327853 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:25.327834    2577 scope.go:117] "RemoveContainer" containerID="f396e793f322043160802136c2ebfb2ffc2a58703703010b502708e0cfead5b5"
May 28 08:38:25.335256 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:25.335237    2577 scope.go:117] "RemoveContainer" containerID="7e3bdc307ae073cd276f816580bfdc3199aa6b502e8329cf95d71c98011c2fd7"
May 28 08:38:25.344996 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:25.344964    2577 scope.go:117] "RemoveContainer" containerID="5a0dbb008a7ab668699d517e37110d5c978fbcbce8ab091cc6b39171231f3b4d"
May 28 08:38:25.347171 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:25.347149    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"]
May 28 08:38:25.350703 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:25.350650    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-66lkc"]
May 28 08:38:25.354056 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:25.354038    2577 scope.go:117] "RemoveContainer" containerID="cbb1f8b196e3ced685043991a8624abec1b43ea556b659743e83e4e7769220a5"
May 28 08:38:25.361400 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:25.361382    2577 scope.go:117] "RemoveContainer" containerID="93aff9731dabfb4a8bdb419d9da74c3c658b7086dbe232e617e0a2b467817a5e"
May 28 08:38:26.241813 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:38:26.241782    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" path="/var/lib/kubelet/pods/c7ee7d6d-2b50-4618-91d8-a7f5acac7610/volumes"
May 28 08:39:15.680158 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680107    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"]
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680504    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="feast-init"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680518    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="feast-init"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680527    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="offline"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680533    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="offline"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680548    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="online"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680558    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="online"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680569    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="feast-apply"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680574    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="feast-apply"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680582    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="ui"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680587    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="ui"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680594    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="registry"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680599    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="registry"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680686    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="offline"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680695    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="online"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680703    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="ui"
May 28 08:39:15.680772 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.680710    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="c7ee7d6d-2b50-4618-91d8-a7f5acac7610" containerName="registry"
May 28 08:39:15.685407 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.685388    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.688603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.688392    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-online-tls\""
May 28 08:39:15.688603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.688420    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-client-ca\""
May 28 08:39:15.688603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.688429    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"kube-root-ca.crt\""
May 28 08:39:15.688603 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.688473    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-ui-tls\""
May 28 08:39:15.688907 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.688741    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"test-ns-remote-registry\"/\"openshift-service-ca.crt\""
May 28 08:39:15.688907 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.688806    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-offline-tls\""
May 28 08:39:15.689622 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.689604    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"test-ns-remote-registry\"/\"feast-simple-feast-remote-setup-dockercfg-gn8cz\""
May 28 08:39:15.697225 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.697202    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"]
May 28 08:39:15.776546 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.776510    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.776546 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.776562    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.776846 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.776607    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/88802884-e873-46aa-8752-42f3eb753964-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.776846 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.776630    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/88802884-e873-46aa-8752-42f3eb753964-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.776846 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.776684    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-p9q25\" (UniqueName: \"kubernetes.io/projected/88802884-e873-46aa-8752-42f3eb753964-kube-api-access-p9q25\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.776846 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.776737    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.877428 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.877395    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/88802884-e873-46aa-8752-42f3eb753964-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.877428 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.877437    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/88802884-e873-46aa-8752-42f3eb753964-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.877751 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.877469    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-p9q25\" (UniqueName: \"kubernetes.io/projected/88802884-e873-46aa-8752-42f3eb753964-kube-api-access-p9q25\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.877751 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.877512    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.877751 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.877584    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.877751 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.877619    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.877751 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:39:15.877737    2577 secret.go:189] Couldn't get secret test-ns-remote-registry/feast-simple-feast-remote-setup-offline-tls: secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 08:39:15.878028 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:39:15.877808    2577 nestedpendingoperations.go:348] Operation for "{volumeName:kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-offline-tls podName:88802884-e873-46aa-8752-42f3eb753964 nodeName:}" failed. No retries permitted until 2026-05-28 08:39:16.377789224 +0000 UTC m=+1634.813968381 (durationBeforeRetry 500ms). Error: MountVolume.SetUp failed for volume "offline-tls" (UniqueName: "kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-offline-tls") pod "feast-simple-feast-remote-setup-5cff94746b-mzbbc" (UID: "88802884-e873-46aa-8752-42f3eb753964") : secret "feast-simple-feast-remote-setup-offline-tls" not found
May 28 08:39:15.878028 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.877881    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/88802884-e873-46aa-8752-42f3eb753964-feast-data\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.878186 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.878167    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/88802884-e873-46aa-8752-42f3eb753964-registry-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.880396 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.880373    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-online-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.880524 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.880508    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-ui-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:15.887143 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:15.887117    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-p9q25\" (UniqueName: \"kubernetes.io/projected/88802884-e873-46aa-8752-42f3eb753964-kube-api-access-p9q25\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:16.381412 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:16.381372    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:16.383929 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:16.383910    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-offline-tls\") pod \"feast-simple-feast-remote-setup-5cff94746b-mzbbc\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") " pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:16.596195 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:16.596154    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:16.728358 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:16.728332    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"]
May 28 08:39:16.730084 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:39:16.730057    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod88802884_e873_46aa_8752_42f3eb753964.slice/crio-0ead7dd24d83fb2a4a6d8e70a745ce3bacbca94a049726a4e38c485fa468c7f8 WatchSource:0}: Error finding container 0ead7dd24d83fb2a4a6d8e70a745ce3bacbca94a049726a4e38c485fa468c7f8: Status 404 returned error can't find the container with id 0ead7dd24d83fb2a4a6d8e70a745ce3bacbca94a049726a4e38c485fa468c7f8
May 28 08:39:17.472896 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:17.472856    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" event={"ID":"88802884-e873-46aa-8752-42f3eb753964","Type":"ContainerStarted","Data":"14bbfa1004d52e5dd6765953174768172879049fec452245b520e032ce5f100c"}
May 28 08:39:17.472896 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:17.472891    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" event={"ID":"88802884-e873-46aa-8752-42f3eb753964","Type":"ContainerStarted","Data":"0ead7dd24d83fb2a4a6d8e70a745ce3bacbca94a049726a4e38c485fa468c7f8"}
May 28 08:39:21.486228 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:21.486193    2577 generic.go:358] "Generic (PLEG): container finished" podID="88802884-e873-46aa-8752-42f3eb753964" containerID="14bbfa1004d52e5dd6765953174768172879049fec452245b520e032ce5f100c" exitCode=0
May 28 08:39:21.486777 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:21.486269    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" event={"ID":"88802884-e873-46aa-8752-42f3eb753964","Type":"ContainerDied","Data":"14bbfa1004d52e5dd6765953174768172879049fec452245b520e032ce5f100c"}
May 28 08:39:22.490146 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:22.490111    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" event={"ID":"88802884-e873-46aa-8752-42f3eb753964","Type":"ContainerStarted","Data":"b4fb18a3a10d2d815e5a5a585dcdf535c2bed3ff864f9c6f9d55549f2af37b31"}
May 28 08:39:27.505488 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:27.505402    2577 generic.go:358] "Generic (PLEG): container finished" podID="88802884-e873-46aa-8752-42f3eb753964" containerID="b4fb18a3a10d2d815e5a5a585dcdf535c2bed3ff864f9c6f9d55549f2af37b31" exitCode=0
May 28 08:39:27.505488 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:27.505455    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" event={"ID":"88802884-e873-46aa-8752-42f3eb753964","Type":"ContainerDied","Data":"b4fb18a3a10d2d815e5a5a585dcdf535c2bed3ff864f9c6f9d55549f2af37b31"}
May 28 08:39:28.512309 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:28.512273    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" event={"ID":"88802884-e873-46aa-8752-42f3eb753964","Type":"ContainerStarted","Data":"fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06"}
May 28 08:39:28.512820 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:28.512319    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" event={"ID":"88802884-e873-46aa-8752-42f3eb753964","Type":"ContainerStarted","Data":"290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae"}
May 28 08:39:28.512820 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:28.512330    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" event={"ID":"88802884-e873-46aa-8752-42f3eb753964","Type":"ContainerStarted","Data":"39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e"}
May 28 08:39:28.540324 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:28.540261    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podStartSLOduration=13.540241048 podStartE2EDuration="13.540241048s" podCreationTimestamp="2026-05-28 08:39:15 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:39:28.538328416 +0000 UTC m=+1646.974507584" watchObservedRunningTime="2026-05-28 08:39:28.540241048 +0000 UTC m=+1646.976420214"
May 28 08:39:28.596327 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:28.596280    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:28.596327 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:28.596329    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:28.596553 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:28.596345    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="unhealthy" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:28.598565 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:28.598519    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.38:8816: connect: connection refused"
May 28 08:39:28.598565 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:28.598539    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 28 08:39:28.598806 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:28.598587    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 28 08:39:31.596548 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:31.596493    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 28 08:39:31.597072 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:31.596496    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="offline" probeResult="failure" output="dial tcp 10.134.0.38:8816: connect: connection refused"
May 28 08:39:31.597072 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:31.596507    2577 prober.go:120] "Probe failed" probeType="Startup" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 28 08:39:34.597919 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:34.597888    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:34.598356 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:34.597974    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:34.598356 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:34.598345    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:34.598486 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:34.598378    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:34.598613 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:34.598593    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:34.598790 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:34.598774    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:34.601746 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:34.601731    2577 kubelet.go:2658] "SyncLoop (probe)" probe="startup" status="started" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:35.534353 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:35.534321    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:39:35.537406 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:39:35.537384    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:40:36.799154 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:36.799117    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"]
May 28 08:40:36.799731 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:36.799572    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="online" containerID="cri-o://39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e" gracePeriod=30
May 28 08:40:36.799731 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:36.799674    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="ui" containerID="cri-o://fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06" gracePeriod=30
May 28 08:40:36.799878 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:36.799734    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="offline" containerID="cri-o://290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae" gracePeriod=30
May 28 08:40:37.047428 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:37.047379    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"]
May 28 08:40:37.047918 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:37.047799    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="registry" containerID="cri-o://58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206" gracePeriod=30
May 28 08:40:37.047918 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:37.047827    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="ui" containerID="cri-o://8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5" gracePeriod=30
May 28 08:40:37.047918 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:37.047883    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="offline" containerID="cri-o://3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072" gracePeriod=30
May 28 08:40:37.047918 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:37.047864    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="online" containerID="cri-o://3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d" gracePeriod=30
May 28 08:40:37.722933 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:37.722902    2577 generic.go:358] "Generic (PLEG): container finished" podID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerID="8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5" exitCode=0
May 28 08:40:37.723096 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:37.722973    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerDied","Data":"8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5"}
May 28 08:40:37.725434 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:37.725412    2577 generic.go:358] "Generic (PLEG): container finished" podID="88802884-e873-46aa-8752-42f3eb753964" containerID="fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06" exitCode=0
May 28 08:40:37.725434 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:37.725430    2577 generic.go:358] "Generic (PLEG): container finished" podID="88802884-e873-46aa-8752-42f3eb753964" containerID="39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e" exitCode=0
May 28 08:40:37.725590 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:37.725479    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" event={"ID":"88802884-e873-46aa-8752-42f3eb753964","Type":"ContainerDied","Data":"fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06"}
May 28 08:40:37.725590 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:37.725505    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" event={"ID":"88802884-e873-46aa-8752-42f3eb753964","Type":"ContainerDied","Data":"39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e"}
May 28 08:40:38.730840 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:38.730801    2577 generic.go:358] "Generic (PLEG): container finished" podID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerID="3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d" exitCode=0
May 28 08:40:38.731218 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:38.730859    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerDied","Data":"3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d"}
May 28 08:40:43.277983 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:43.277932    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 28 08:40:43.278472 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:43.278022    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 28 08:40:44.598817 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:44.598757    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 28 08:40:45.535148 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:45.535099    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 28 08:40:53.278706 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:53.278640    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 28 08:40:53.278706 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:53.278693    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 28 08:40:54.598876 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:54.598833    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 28 08:40:55.534815 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:40:55.534772    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 28 08:41:03.278602 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:03.278553    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.37:8443: connect: connection refused"
May 28 08:41:03.279112 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:03.278565    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="online" probeResult="failure" output="Get \"https://10.134.0.37:6567/health\": dial tcp 10.134.0.37:6567: connect: connection refused"
May 28 08:41:03.279112 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:03.278742    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:41:03.279112 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:03.278795    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:41:04.598683 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:04.598608    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="ui" probeResult="failure" output="dial tcp 10.134.0.38:8443: connect: connection refused"
May 28 08:41:04.599090 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:04.598776    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:41:05.534582 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:05.534538    2577 prober.go:120] "Probe failed" probeType="Readiness" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="online" probeResult="failure" output="Get \"https://10.134.0.38:6567/health\": dial tcp 10.134.0.38:6567: connect: connection refused"
May 28 08:41:05.534789 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:05.534717    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:41:07.449983 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.449942    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:41:07.540546 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.540500    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-offline-tls\") pod \"88802884-e873-46aa-8752-42f3eb753964\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") "
May 28 08:41:07.540546 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.540551    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-p9q25\" (UniqueName: \"kubernetes.io/projected/88802884-e873-46aa-8752-42f3eb753964-kube-api-access-p9q25\") pod \"88802884-e873-46aa-8752-42f3eb753964\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") "
May 28 08:41:07.540841 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.540571    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/88802884-e873-46aa-8752-42f3eb753964-registry-tls\") pod \"88802884-e873-46aa-8752-42f3eb753964\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") "
May 28 08:41:07.540841 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.540594    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/88802884-e873-46aa-8752-42f3eb753964-feast-data\") pod \"88802884-e873-46aa-8752-42f3eb753964\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") "
May 28 08:41:07.540841 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.540720    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-ui-tls\") pod \"88802884-e873-46aa-8752-42f3eb753964\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") "
May 28 08:41:07.540841 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.540782    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-online-tls\") pod \"88802884-e873-46aa-8752-42f3eb753964\" (UID: \"88802884-e873-46aa-8752-42f3eb753964\") "
May 28 08:41:07.541050 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.541027    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/configmap/88802884-e873-46aa-8752-42f3eb753964-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "88802884-e873-46aa-8752-42f3eb753964" (UID: "88802884-e873-46aa-8752-42f3eb753964"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/configmap", VolumeGIDValue ""
May 28 08:41:07.541209 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.541176    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/88802884-e873-46aa-8752-42f3eb753964-feast-data" (OuterVolumeSpecName: "feast-data") pod "88802884-e873-46aa-8752-42f3eb753964" (UID: "88802884-e873-46aa-8752-42f3eb753964"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:41:07.543752 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.543716    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/88802884-e873-46aa-8752-42f3eb753964-kube-api-access-p9q25" (OuterVolumeSpecName: "kube-api-access-p9q25") pod "88802884-e873-46aa-8752-42f3eb753964" (UID: "88802884-e873-46aa-8752-42f3eb753964"). InnerVolumeSpecName "kube-api-access-p9q25". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:41:07.543752 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.543740    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-online-tls" (OuterVolumeSpecName: "online-tls") pod "88802884-e873-46aa-8752-42f3eb753964" (UID: "88802884-e873-46aa-8752-42f3eb753964"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:41:07.543752 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.543724    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "88802884-e873-46aa-8752-42f3eb753964" (UID: "88802884-e873-46aa-8752-42f3eb753964"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:41:07.543914 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.543772    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "88802884-e873-46aa-8752-42f3eb753964" (UID: "88802884-e873-46aa-8752-42f3eb753964"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:41:07.641993 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.641962    2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-online-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:07.641993 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.641992    2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-offline-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:07.642229 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.642006    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-p9q25\" (UniqueName: \"kubernetes.io/projected/88802884-e873-46aa-8752-42f3eb753964-kube-api-access-p9q25\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:07.642229 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.642018    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/configmap/88802884-e873-46aa-8752-42f3eb753964-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:07.642229 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.642030    2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/88802884-e873-46aa-8752-42f3eb753964-feast-data\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:07.642229 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.642040    2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/88802884-e873-46aa-8752-42f3eb753964-ui-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:07.698551 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.698527    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:41:07.742949 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.742918    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-ui-tls\") pod \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") "
May 28 08:41:07.743137 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.742960    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-offline-tls\") pod \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") "
May 28 08:41:07.743137 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.743033    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-mpx4b\" (UniqueName: \"kubernetes.io/projected/a4aff123-3b0a-4245-ad33-bd3923ebc30f-kube-api-access-mpx4b\") pod \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") "
May 28 08:41:07.743137 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.743063    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-online-tls\") pod \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") "
May 28 08:41:07.743137 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.743110    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-registry-tls\") pod \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") "
May 28 08:41:07.743320 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.743140    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a4aff123-3b0a-4245-ad33-bd3923ebc30f-feast-data\") pod \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\" (UID: \"a4aff123-3b0a-4245-ad33-bd3923ebc30f\") "
May 28 08:41:07.743835 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.743804    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/a4aff123-3b0a-4245-ad33-bd3923ebc30f-feast-data" (OuterVolumeSpecName: "feast-data") pod "a4aff123-3b0a-4245-ad33-bd3923ebc30f" (UID: "a4aff123-3b0a-4245-ad33-bd3923ebc30f"). InnerVolumeSpecName "feast-data". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:41:07.745911 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.745705    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-ui-tls" (OuterVolumeSpecName: "ui-tls") pod "a4aff123-3b0a-4245-ad33-bd3923ebc30f" (UID: "a4aff123-3b0a-4245-ad33-bd3923ebc30f"). InnerVolumeSpecName "ui-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:41:07.745911 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.745809    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-online-tls" (OuterVolumeSpecName: "online-tls") pod "a4aff123-3b0a-4245-ad33-bd3923ebc30f" (UID: "a4aff123-3b0a-4245-ad33-bd3923ebc30f"). InnerVolumeSpecName "online-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:41:07.745911 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.745827    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-offline-tls" (OuterVolumeSpecName: "offline-tls") pod "a4aff123-3b0a-4245-ad33-bd3923ebc30f" (UID: "a4aff123-3b0a-4245-ad33-bd3923ebc30f"). InnerVolumeSpecName "offline-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:41:07.746308 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.746280    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/a4aff123-3b0a-4245-ad33-bd3923ebc30f-kube-api-access-mpx4b" (OuterVolumeSpecName: "kube-api-access-mpx4b") pod "a4aff123-3b0a-4245-ad33-bd3923ebc30f" (UID: "a4aff123-3b0a-4245-ad33-bd3923ebc30f"). InnerVolumeSpecName "kube-api-access-mpx4b". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:41:07.746398 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.746377    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-registry-tls" (OuterVolumeSpecName: "registry-tls") pod "a4aff123-3b0a-4245-ad33-bd3923ebc30f" (UID: "a4aff123-3b0a-4245-ad33-bd3923ebc30f"). InnerVolumeSpecName "registry-tls". PluginName "kubernetes.io/secret", VolumeGIDValue ""
May 28 08:41:07.818936 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.818902    2577 generic.go:358] "Generic (PLEG): container finished" podID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerID="3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072" exitCode=137
May 28 08:41:07.818936 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.818934    2577 generic.go:358] "Generic (PLEG): container finished" podID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerID="58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206" exitCode=137
May 28 08:41:07.819164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.818998    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"
May 28 08:41:07.819164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.818987    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerDied","Data":"3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072"}
May 28 08:41:07.819164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.819140    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerDied","Data":"58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206"}
May 28 08:41:07.819164 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.819159    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9" event={"ID":"a4aff123-3b0a-4245-ad33-bd3923ebc30f","Type":"ContainerDied","Data":"224ca5529722fc456349c2dc832e0eacaa9b8f197f90a1e994c45806d5930b8a"}
May 28 08:41:07.819359 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.819180    2577 scope.go:117] "RemoveContainer" containerID="8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5"
May 28 08:41:07.821691 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.821638    2577 generic.go:358] "Generic (PLEG): container finished" podID="88802884-e873-46aa-8752-42f3eb753964" containerID="290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae" exitCode=137
May 28 08:41:07.821832 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.821736    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"
May 28 08:41:07.821832 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.821735    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" event={"ID":"88802884-e873-46aa-8752-42f3eb753964","Type":"ContainerDied","Data":"290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae"}
May 28 08:41:07.821962 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.821848    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc" event={"ID":"88802884-e873-46aa-8752-42f3eb753964","Type":"ContainerDied","Data":"0ead7dd24d83fb2a4a6d8e70a745ce3bacbca94a049726a4e38c485fa468c7f8"}
May 28 08:41:07.829829 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.829809    2577 scope.go:117] "RemoveContainer" containerID="3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072"
May 28 08:41:07.838160 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.838141    2577 scope.go:117] "RemoveContainer" containerID="3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d"
May 28 08:41:07.843943 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.843924    2577 reconciler_common.go:299] "Volume detached for volume \"online-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-online-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:07.844027 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.843950    2577 reconciler_common.go:299] "Volume detached for volume \"registry-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-registry-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:07.844027 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.843960    2577 reconciler_common.go:299] "Volume detached for volume \"feast-data\" (UniqueName: \"kubernetes.io/empty-dir/a4aff123-3b0a-4245-ad33-bd3923ebc30f-feast-data\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:07.844027 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.843968    2577 reconciler_common.go:299] "Volume detached for volume \"ui-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-ui-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:07.844027 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.843978    2577 reconciler_common.go:299] "Volume detached for volume \"offline-tls\" (UniqueName: \"kubernetes.io/secret/a4aff123-3b0a-4245-ad33-bd3923ebc30f-offline-tls\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:07.844027 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.843986    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-mpx4b\" (UniqueName: \"kubernetes.io/projected/a4aff123-3b0a-4245-ad33-bd3923ebc30f-kube-api-access-mpx4b\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:07.846188 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.846169    2577 scope.go:117] "RemoveContainer" containerID="58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206"
May 28 08:41:07.847357 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.847337    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"]
May 28 08:41:07.850939 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.850918    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-feast/feast-simple-feast-setup-664fcfcc58-lmhp9"]
May 28 08:41:07.854183 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.854163    2577 scope.go:117] "RemoveContainer" containerID="8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636"
May 28 08:41:07.861443 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.861421    2577 scope.go:117] "RemoveContainer" containerID="3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a"
May 28 08:41:07.864998 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.864971    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"]
May 28 08:41:07.868986 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.868936    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["test-ns-remote-registry/feast-simple-feast-remote-setup-5cff94746b-mzbbc"]
May 28 08:41:07.869646 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.869630    2577 scope.go:117] "RemoveContainer" containerID="8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5"
May 28 08:41:07.869951 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:07.869928    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5\": container with ID starting with 8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5 not found: ID does not exist" containerID="8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5"
May 28 08:41:07.870051 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.869958    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5"} err="failed to get container status \"8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5\": rpc error: code = NotFound desc = could not find container \"8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5\": container with ID starting with 8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5 not found: ID does not exist"
May 28 08:41:07.870051 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.869979    2577 scope.go:117] "RemoveContainer" containerID="3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072"
May 28 08:41:07.870243 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:07.870226    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072\": container with ID starting with 3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072 not found: ID does not exist" containerID="3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072"
May 28 08:41:07.870289 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.870248    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072"} err="failed to get container status \"3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072\": rpc error: code = NotFound desc = could not find container \"3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072\": container with ID starting with 3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072 not found: ID does not exist"
May 28 08:41:07.870289 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.870262    2577 scope.go:117] "RemoveContainer" containerID="3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d"
May 28 08:41:07.870478 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:07.870462    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d\": container with ID starting with 3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d not found: ID does not exist" containerID="3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d"
May 28 08:41:07.870549 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.870497    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d"} err="failed to get container status \"3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d\": rpc error: code = NotFound desc = could not find container \"3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d\": container with ID starting with 3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d not found: ID does not exist"
May 28 08:41:07.870549 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.870520    2577 scope.go:117] "RemoveContainer" containerID="58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206"
May 28 08:41:07.870818 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:07.870797    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206\": container with ID starting with 58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206 not found: ID does not exist" containerID="58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206"
May 28 08:41:07.870875 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.870824    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206"} err="failed to get container status \"58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206\": rpc error: code = NotFound desc = could not find container \"58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206\": container with ID starting with 58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206 not found: ID does not exist"
May 28 08:41:07.870875 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.870842    2577 scope.go:117] "RemoveContainer" containerID="8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636"
May 28 08:41:07.871067 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:07.871051    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636\": container with ID starting with 8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636 not found: ID does not exist" containerID="8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636"
May 28 08:41:07.871110 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.871071    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636"} err="failed to get container status \"8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636\": rpc error: code = NotFound desc = could not find container \"8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636\": container with ID starting with 8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636 not found: ID does not exist"
May 28 08:41:07.871110 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.871095    2577 scope.go:117] "RemoveContainer" containerID="3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a"
May 28 08:41:07.871307 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:07.871291    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a\": container with ID starting with 3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a not found: ID does not exist" containerID="3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a"
May 28 08:41:07.871382 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.871316    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a"} err="failed to get container status \"3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a\": rpc error: code = NotFound desc = could not find container \"3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a\": container with ID starting with 3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a not found: ID does not exist"
May 28 08:41:07.871382 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.871336    2577 scope.go:117] "RemoveContainer" containerID="8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5"
May 28 08:41:07.871568 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.871551    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5"} err="failed to get container status \"8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5\": rpc error: code = NotFound desc = could not find container \"8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5\": container with ID starting with 8319fe6cfb93b969a5bc67cbb4e98890317783195e9a1c097bb4a42721fb2db5 not found: ID does not exist"
May 28 08:41:07.871633 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.871569    2577 scope.go:117] "RemoveContainer" containerID="3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072"
May 28 08:41:07.871808 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.871788    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072"} err="failed to get container status \"3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072\": rpc error: code = NotFound desc = could not find container \"3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072\": container with ID starting with 3d23e1ec06f0c3c42836345ceefcab3d78ca931c28945de115e099b5e2e17072 not found: ID does not exist"
May 28 08:41:07.871860 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.871809    2577 scope.go:117] "RemoveContainer" containerID="3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d"
May 28 08:41:07.872014 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.871991    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d"} err="failed to get container status \"3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d\": rpc error: code = NotFound desc = could not find container \"3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d\": container with ID starting with 3cfc266d407ca782b52ab7bfaf32991ef898d5ae058f09310d4ffac2814d6d4d not found: ID does not exist"
May 28 08:41:07.872096 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.872017    2577 scope.go:117] "RemoveContainer" containerID="58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206"
May 28 08:41:07.872211 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.872196    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206"} err="failed to get container status \"58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206\": rpc error: code = NotFound desc = could not find container \"58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206\": container with ID starting with 58e283eaaadfc9efb5a7a797e81a73dc82a4899c50d4142c3ba9ea65fc40d206 not found: ID does not exist"
May 28 08:41:07.872271 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.872212    2577 scope.go:117] "RemoveContainer" containerID="8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636"
May 28 08:41:07.872407 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.872393    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636"} err="failed to get container status \"8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636\": rpc error: code = NotFound desc = could not find container \"8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636\": container with ID starting with 8a4ab609ba0d5dffe157a3a1242fd09823bd41c360d5761957410aee3aed6636 not found: ID does not exist"
May 28 08:41:07.872455 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.872408    2577 scope.go:117] "RemoveContainer" containerID="3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a"
May 28 08:41:07.872586 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.872569    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a"} err="failed to get container status \"3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a\": rpc error: code = NotFound desc = could not find container \"3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a\": container with ID starting with 3c082c0e6d5fe129ae5bc897eee3a26ea0a7b9f3aba1d1c9aac0d07756d2ac9a not found: ID does not exist"
May 28 08:41:07.872678 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.872588    2577 scope.go:117] "RemoveContainer" containerID="fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06"
May 28 08:41:07.880250 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.880235    2577 scope.go:117] "RemoveContainer" containerID="290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae"
May 28 08:41:07.887247 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.887224    2577 scope.go:117] "RemoveContainer" containerID="39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e"
May 28 08:41:07.894377 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.894359    2577 scope.go:117] "RemoveContainer" containerID="b4fb18a3a10d2d815e5a5a585dcdf535c2bed3ff864f9c6f9d55549f2af37b31"
May 28 08:41:07.901274 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.901252    2577 scope.go:117] "RemoveContainer" containerID="14bbfa1004d52e5dd6765953174768172879049fec452245b520e032ce5f100c"
May 28 08:41:07.908399 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.908381    2577 scope.go:117] "RemoveContainer" containerID="fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06"
May 28 08:41:07.908638 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:07.908618    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06\": container with ID starting with fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06 not found: ID does not exist" containerID="fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06"
May 28 08:41:07.908750 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.908644    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06"} err="failed to get container status \"fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06\": rpc error: code = NotFound desc = could not find container \"fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06\": container with ID starting with fcef389939276cad7a8d5d5d39b6c4366c4289181b98a12b9e2078289d162b06 not found: ID does not exist"
May 28 08:41:07.908750 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.908682    2577 scope.go:117] "RemoveContainer" containerID="290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae"
May 28 08:41:07.908921 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:07.908903    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae\": container with ID starting with 290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae not found: ID does not exist" containerID="290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae"
May 28 08:41:07.908964 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.908929    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae"} err="failed to get container status \"290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae\": rpc error: code = NotFound desc = could not find container \"290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae\": container with ID starting with 290e009561ea100035427ab57c956350c2e8abaa9edee2692c0f221b9a9e79ae not found: ID does not exist"
May 28 08:41:07.908964 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.908946    2577 scope.go:117] "RemoveContainer" containerID="39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e"
May 28 08:41:07.909199 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:07.909181    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e\": container with ID starting with 39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e not found: ID does not exist" containerID="39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e"
May 28 08:41:07.909255 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.909208    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e"} err="failed to get container status \"39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e\": rpc error: code = NotFound desc = could not find container \"39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e\": container with ID starting with 39273c2a4701403a87f0c70b42ee825a65f6ad408988d3bbdaa1886be18aae2e not found: ID does not exist"
May 28 08:41:07.909255 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.909225    2577 scope.go:117] "RemoveContainer" containerID="b4fb18a3a10d2d815e5a5a585dcdf535c2bed3ff864f9c6f9d55549f2af37b31"
May 28 08:41:07.909432 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:07.909413    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b4fb18a3a10d2d815e5a5a585dcdf535c2bed3ff864f9c6f9d55549f2af37b31\": container with ID starting with b4fb18a3a10d2d815e5a5a585dcdf535c2bed3ff864f9c6f9d55549f2af37b31 not found: ID does not exist" containerID="b4fb18a3a10d2d815e5a5a585dcdf535c2bed3ff864f9c6f9d55549f2af37b31"
May 28 08:41:07.909475 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.909438    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b4fb18a3a10d2d815e5a5a585dcdf535c2bed3ff864f9c6f9d55549f2af37b31"} err="failed to get container status \"b4fb18a3a10d2d815e5a5a585dcdf535c2bed3ff864f9c6f9d55549f2af37b31\": rpc error: code = NotFound desc = could not find container \"b4fb18a3a10d2d815e5a5a585dcdf535c2bed3ff864f9c6f9d55549f2af37b31\": container with ID starting with b4fb18a3a10d2d815e5a5a585dcdf535c2bed3ff864f9c6f9d55549f2af37b31 not found: ID does not exist"
May 28 08:41:07.909475 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.909450    2577 scope.go:117] "RemoveContainer" containerID="14bbfa1004d52e5dd6765953174768172879049fec452245b520e032ce5f100c"
May 28 08:41:07.909699 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:07.909680    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"14bbfa1004d52e5dd6765953174768172879049fec452245b520e032ce5f100c\": container with ID starting with 14bbfa1004d52e5dd6765953174768172879049fec452245b520e032ce5f100c not found: ID does not exist" containerID="14bbfa1004d52e5dd6765953174768172879049fec452245b520e032ce5f100c"
May 28 08:41:07.909758 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:07.909704    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"14bbfa1004d52e5dd6765953174768172879049fec452245b520e032ce5f100c"} err="failed to get container status \"14bbfa1004d52e5dd6765953174768172879049fec452245b520e032ce5f100c\": rpc error: code = NotFound desc = could not find container \"14bbfa1004d52e5dd6765953174768172879049fec452245b520e032ce5f100c\": container with ID starting with 14bbfa1004d52e5dd6765953174768172879049fec452245b520e032ce5f100c not found: ID does not exist"
May 28 08:41:08.242061 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:08.241980    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="88802884-e873-46aa-8752-42f3eb753964" path="/var/lib/kubelet/pods/88802884-e873-46aa-8752-42f3eb753964/volumes"
May 28 08:41:08.242523 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:08.242510    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" path="/var/lib/kubelet/pods/a4aff123-3b0a-4245-ad33-bd3923ebc30f/volumes"
May 28 08:41:26.480420 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480384    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-kspvx/must-gather-h2f72"]
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480855    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="registry"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480875    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="registry"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480890    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="feast-init"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480899    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="feast-init"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480913    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="offline"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480921    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="offline"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480933    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="feast-apply"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480942    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="feast-apply"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480950    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="online"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480958    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="online"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480967    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="feast-apply"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480975    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="feast-apply"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480989    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="online"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.480995    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="online"
May 28 08:41:26.481002 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481006    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="offline"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481014    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="offline"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481024    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="ui"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481032    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="ui"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481045    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="ui"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481053    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="ui"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481062    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="feast-init"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481070    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="feast-init"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481144    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="offline"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481160    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="online"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481171    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="ui"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481181    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="offline"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481192    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="ui"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481203    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="88802884-e873-46aa-8752-42f3eb753964" containerName="online"
May 28 08:41:26.481787 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.481215    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="a4aff123-3b0a-4245-ad33-bd3923ebc30f" containerName="registry"
May 28 08:41:26.486328 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.486309    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kspvx/must-gather-h2f72"
May 28 08:41:26.489356 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.489335    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kspvx\"/\"kube-root-ca.crt\""
May 28 08:41:26.490767 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.490743    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-kspvx\"/\"openshift-service-ca.crt\""
May 28 08:41:26.490894 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.490824    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-kspvx\"/\"default-dockercfg-z28r8\""
May 28 08:41:26.492510 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.492488    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kspvx/must-gather-h2f72"]
May 28 08:41:26.600440 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.600396    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/818596ea-a57a-48b1-8e83-e7e67e1dcd0c-must-gather-output\") pod \"must-gather-h2f72\" (UID: \"818596ea-a57a-48b1-8e83-e7e67e1dcd0c\") " pod="openshift-must-gather-kspvx/must-gather-h2f72"
May 28 08:41:26.600440 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.600444    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-kz87h\" (UniqueName: \"kubernetes.io/projected/818596ea-a57a-48b1-8e83-e7e67e1dcd0c-kube-api-access-kz87h\") pod \"must-gather-h2f72\" (UID: \"818596ea-a57a-48b1-8e83-e7e67e1dcd0c\") " pod="openshift-must-gather-kspvx/must-gather-h2f72"
May 28 08:41:26.701862 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.701827    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/818596ea-a57a-48b1-8e83-e7e67e1dcd0c-must-gather-output\") pod \"must-gather-h2f72\" (UID: \"818596ea-a57a-48b1-8e83-e7e67e1dcd0c\") " pod="openshift-must-gather-kspvx/must-gather-h2f72"
May 28 08:41:26.701862 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.701863    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-kz87h\" (UniqueName: \"kubernetes.io/projected/818596ea-a57a-48b1-8e83-e7e67e1dcd0c-kube-api-access-kz87h\") pod \"must-gather-h2f72\" (UID: \"818596ea-a57a-48b1-8e83-e7e67e1dcd0c\") " pod="openshift-must-gather-kspvx/must-gather-h2f72"
May 28 08:41:26.702168 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.702147    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/818596ea-a57a-48b1-8e83-e7e67e1dcd0c-must-gather-output\") pod \"must-gather-h2f72\" (UID: \"818596ea-a57a-48b1-8e83-e7e67e1dcd0c\") " pod="openshift-must-gather-kspvx/must-gather-h2f72"
May 28 08:41:26.710971 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.710948    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-kz87h\" (UniqueName: \"kubernetes.io/projected/818596ea-a57a-48b1-8e83-e7e67e1dcd0c-kube-api-access-kz87h\") pod \"must-gather-h2f72\" (UID: \"818596ea-a57a-48b1-8e83-e7e67e1dcd0c\") " pod="openshift-must-gather-kspvx/must-gather-h2f72"
May 28 08:41:26.796132 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.796099    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kspvx/must-gather-h2f72"
May 28 08:41:26.928797 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:26.928766    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-kspvx/must-gather-h2f72"]
May 28 08:41:26.932755 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:41:26.932722    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-besteffort.slice/kubepods-besteffort-pod818596ea_a57a_48b1_8e83_e7e67e1dcd0c.slice/crio-6ad7b21fbde394b9dae3c3d78a073ed99368813ddc18a0285ab569beda7278cf WatchSource:0}: Error finding container 6ad7b21fbde394b9dae3c3d78a073ed99368813ddc18a0285ab569beda7278cf: Status 404 returned error can't find the container with id 6ad7b21fbde394b9dae3c3d78a073ed99368813ddc18a0285ab569beda7278cf
May 28 08:41:27.887456 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:27.887394    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kspvx/must-gather-h2f72" event={"ID":"818596ea-a57a-48b1-8e83-e7e67e1dcd0c","Type":"ContainerStarted","Data":"6ad7b21fbde394b9dae3c3d78a073ed99368813ddc18a0285ab569beda7278cf"}
May 28 08:41:33.909834 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:33.909790    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kspvx/must-gather-h2f72" event={"ID":"818596ea-a57a-48b1-8e83-e7e67e1dcd0c","Type":"ContainerStarted","Data":"b17f98cad86aee279dc1e62a905537705e7ffd3b62a28a2fc0c4ad0302b2b357"}
May 28 08:41:33.910204 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:33.909841    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kspvx/must-gather-h2f72" event={"ID":"818596ea-a57a-48b1-8e83-e7e67e1dcd0c","Type":"ContainerStarted","Data":"45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9"}
May 28 08:41:33.926945 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:33.926875    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-kspvx/must-gather-h2f72" podStartSLOduration=1.774099665 podStartE2EDuration="7.926854808s" podCreationTimestamp="2026-05-28 08:41:26 +0000 UTC" firstStartedPulling="2026-05-28 08:41:26.934377863 +0000 UTC m=+1765.370557006" lastFinishedPulling="2026-05-28 08:41:33.087133 +0000 UTC m=+1771.523312149" observedRunningTime="2026-05-28 08:41:33.926484143 +0000 UTC m=+1772.362663309" watchObservedRunningTime="2026-05-28 08:41:33.926854808 +0000 UTC m=+1772.363033975"
May 28 08:41:43.943208 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:43.943172    2577 generic.go:358] "Generic (PLEG): container finished" podID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" containerID="45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9" exitCode=0
May 28 08:41:43.943749 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:43.943236    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-kspvx/must-gather-h2f72" event={"ID":"818596ea-a57a-48b1-8e83-e7e67e1dcd0c","Type":"ContainerDied","Data":"45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9"}
May 28 08:41:43.943749 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:43.943637    2577 scope.go:117] "RemoveContainer" containerID="45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9"
May 28 08:41:43.975532 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:43.975500    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-kspvx_must-gather-h2f72_818596ea-a57a-48b1-8e83-e7e67e1dcd0c/gather/0.log"
May 28 08:41:47.415485 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:47.415456    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_global-pull-secret-syncer-nwdqc_18b9e331-35ec-44e5-9025-158761237d3d/global-pull-secret-syncer/0.log"
May 28 08:41:47.514474 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:47.514443    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_konnectivity-agent-j8bt4_80faa7ab-5aca-4d3f-96cf-f7edc87643fe/konnectivity-agent/0.log"
May 28 08:41:47.670088 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:47.670004    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/kube-system_kube-apiserver-proxy-ip-10-0-137-158.ec2.internal_ac83f6a6291c1bd1f8450f6fd7149beb/haproxy/0.log"
May 28 08:41:49.326142 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.326107    2577 kubelet.go:2553] "SyncLoop DELETE" source="api" pods=["openshift-must-gather-kspvx/must-gather-h2f72"]
May 28 08:41:49.326536 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.326313    2577 kuberuntime_container.go:864] "Killing container with a grace period" pod="openshift-must-gather-kspvx/must-gather-h2f72" podUID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" containerName="copy" containerID="cri-o://b17f98cad86aee279dc1e62a905537705e7ffd3b62a28a2fc0c4ad0302b2b357" gracePeriod=2
May 28 08:41:49.336151 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.336125    2577 kubelet.go:2547] "SyncLoop REMOVE" source="api" pods=["openshift-must-gather-kspvx/must-gather-h2f72"]
May 28 08:41:49.553348 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.553322    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-kspvx_must-gather-h2f72_818596ea-a57a-48b1-8e83-e7e67e1dcd0c/copy/0.log"
May 28 08:41:49.553703 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.553684    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kspvx/must-gather-h2f72"
May 28 08:41:49.556137 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.556113    2577 status_manager.go:895] "Failed to get status for pod" podUID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" pod="openshift-must-gather-kspvx/must-gather-h2f72" err="pods \"must-gather-h2f72\" is forbidden: User \"system:node:ip-10-0-137-158.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-kspvx\": no relationship found between node 'ip-10-0-137-158.ec2.internal' and this object"
May 28 08:41:49.710696 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.710576    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/818596ea-a57a-48b1-8e83-e7e67e1dcd0c-must-gather-output\") pod \"818596ea-a57a-48b1-8e83-e7e67e1dcd0c\" (UID: \"818596ea-a57a-48b1-8e83-e7e67e1dcd0c\") "
May 28 08:41:49.710696 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.710633    2577 reconciler_common.go:162] "operationExecutor.UnmountVolume started for volume \"kube-api-access-kz87h\" (UniqueName: \"kubernetes.io/projected/818596ea-a57a-48b1-8e83-e7e67e1dcd0c-kube-api-access-kz87h\") pod \"818596ea-a57a-48b1-8e83-e7e67e1dcd0c\" (UID: \"818596ea-a57a-48b1-8e83-e7e67e1dcd0c\") "
May 28 08:41:49.710975 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.710951    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/empty-dir/818596ea-a57a-48b1-8e83-e7e67e1dcd0c-must-gather-output" (OuterVolumeSpecName: "must-gather-output") pod "818596ea-a57a-48b1-8e83-e7e67e1dcd0c" (UID: "818596ea-a57a-48b1-8e83-e7e67e1dcd0c"). InnerVolumeSpecName "must-gather-output". PluginName "kubernetes.io/empty-dir", VolumeGIDValue ""
May 28 08:41:49.712957 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.712928    2577 operation_generator.go:781] UnmountVolume.TearDown succeeded for volume "kubernetes.io/projected/818596ea-a57a-48b1-8e83-e7e67e1dcd0c-kube-api-access-kz87h" (OuterVolumeSpecName: "kube-api-access-kz87h") pod "818596ea-a57a-48b1-8e83-e7e67e1dcd0c" (UID: "818596ea-a57a-48b1-8e83-e7e67e1dcd0c"). InnerVolumeSpecName "kube-api-access-kz87h". PluginName "kubernetes.io/projected", VolumeGIDValue ""
May 28 08:41:49.812265 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.812224    2577 reconciler_common.go:299] "Volume detached for volume \"must-gather-output\" (UniqueName: \"kubernetes.io/empty-dir/818596ea-a57a-48b1-8e83-e7e67e1dcd0c-must-gather-output\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:49.812265 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.812256    2577 reconciler_common.go:299] "Volume detached for volume \"kube-api-access-kz87h\" (UniqueName: \"kubernetes.io/projected/818596ea-a57a-48b1-8e83-e7e67e1dcd0c-kube-api-access-kz87h\") on node \"ip-10-0-137-158.ec2.internal\" DevicePath \"\""
May 28 08:41:49.962925 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.962842    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-must-gather-kspvx_must-gather-h2f72_818596ea-a57a-48b1-8e83-e7e67e1dcd0c/copy/0.log"
May 28 08:41:49.963179 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.963140    2577 generic.go:358] "Generic (PLEG): container finished" podID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" containerID="b17f98cad86aee279dc1e62a905537705e7ffd3b62a28a2fc0c4ad0302b2b357" exitCode=143
May 28 08:41:49.963310 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.963199    2577 util.go:48] "No ready sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-kspvx/must-gather-h2f72"
May 28 08:41:49.963310 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.963210    2577 scope.go:117] "RemoveContainer" containerID="b17f98cad86aee279dc1e62a905537705e7ffd3b62a28a2fc0c4ad0302b2b357"
May 28 08:41:49.966190 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.966164    2577 status_manager.go:895] "Failed to get status for pod" podUID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" pod="openshift-must-gather-kspvx/must-gather-h2f72" err="pods \"must-gather-h2f72\" is forbidden: User \"system:node:ip-10-0-137-158.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-kspvx\": no relationship found between node 'ip-10-0-137-158.ec2.internal' and this object"
May 28 08:41:49.972265 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.972246    2577 scope.go:117] "RemoveContainer" containerID="45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9"
May 28 08:41:49.974909 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.974879    2577 status_manager.go:895] "Failed to get status for pod" podUID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" pod="openshift-must-gather-kspvx/must-gather-h2f72" err="pods \"must-gather-h2f72\" is forbidden: User \"system:node:ip-10-0-137-158.ec2.internal\" cannot get resource \"pods\" in API group \"\" in the namespace \"openshift-must-gather-kspvx\": no relationship found between node 'ip-10-0-137-158.ec2.internal' and this object"
May 28 08:41:49.983170 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.983152    2577 scope.go:117] "RemoveContainer" containerID="b17f98cad86aee279dc1e62a905537705e7ffd3b62a28a2fc0c4ad0302b2b357"
May 28 08:41:49.983455 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:49.983433    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"b17f98cad86aee279dc1e62a905537705e7ffd3b62a28a2fc0c4ad0302b2b357\": container with ID starting with b17f98cad86aee279dc1e62a905537705e7ffd3b62a28a2fc0c4ad0302b2b357 not found: ID does not exist" containerID="b17f98cad86aee279dc1e62a905537705e7ffd3b62a28a2fc0c4ad0302b2b357"
May 28 08:41:49.983512 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.983465    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"b17f98cad86aee279dc1e62a905537705e7ffd3b62a28a2fc0c4ad0302b2b357"} err="failed to get container status \"b17f98cad86aee279dc1e62a905537705e7ffd3b62a28a2fc0c4ad0302b2b357\": rpc error: code = NotFound desc = could not find container \"b17f98cad86aee279dc1e62a905537705e7ffd3b62a28a2fc0c4ad0302b2b357\": container with ID starting with b17f98cad86aee279dc1e62a905537705e7ffd3b62a28a2fc0c4ad0302b2b357 not found: ID does not exist"
May 28 08:41:49.983512 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.983484    2577 scope.go:117] "RemoveContainer" containerID="45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9"
May 28 08:41:49.983770 ip-10-0-137-158 kubenswrapper[2577]: E0528 08:41:49.983752    2577 log.go:32] "ContainerStatus from runtime service failed" err="rpc error: code = NotFound desc = could not find container \"45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9\": container with ID starting with 45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9 not found: ID does not exist" containerID="45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9"
May 28 08:41:49.983833 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:49.983780    2577 pod_container_deletor.go:53] "DeleteContainer returned error" containerID={"Type":"cri-o","ID":"45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9"} err="failed to get container status \"45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9\": rpc error: code = NotFound desc = could not find container \"45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9\": container with ID starting with 45c49e535b9a84098c69fbb9c467ea2658342ee872a57be924c8749c4b76e9a9 not found: ID does not exist"
May 28 08:41:50.242187 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:50.242112    2577 kubelet_volumes.go:163] "Cleaned up orphaned pod volumes dir" podUID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" path="/var/lib/kubelet/pods/818596ea-a57a-48b1-8e83-e7e67e1dcd0c/volumes"
May 28 08:41:50.836292 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:50.836264    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa/alertmanager/0.log"
May 28 08:41:50.863234 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:50.863208    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa/config-reloader/0.log"
May 28 08:41:50.890068 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:50.890034    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa/kube-rbac-proxy-web/0.log"
May 28 08:41:50.917010 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:50.916976    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa/kube-rbac-proxy/0.log"
May 28 08:41:50.942600 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:50.942577    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa/kube-rbac-proxy-metric/0.log"
May 28 08:41:50.968258 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:50.968230    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa/prom-label-proxy/0.log"
May 28 08:41:50.992461 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:50.992434    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_alertmanager-main-0_e3dc6bb1-b7a0-49b5-ac8b-77ec50bd9baa/init-config-reloader/0.log"
May 28 08:41:51.064588 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:51.064560    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-c8cdd_1bb38ee1-d552-4e83-bb65-7ab76b50e910/kube-state-metrics/0.log"
May 28 08:41:51.088870 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:51.088788    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-c8cdd_1bb38ee1-d552-4e83-bb65-7ab76b50e910/kube-rbac-proxy-main/0.log"
May 28 08:41:51.111075 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:51.111047    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_kube-state-metrics-54fd6bc5c4-c8cdd_1bb38ee1-d552-4e83-bb65-7ab76b50e910/kube-rbac-proxy-self/0.log"
May 28 08:41:51.176201 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:51.176173    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_monitoring-plugin-665c9c8584-msdhf_5f6caeea-d618-42f1-a51d-3f6c69aef481/monitoring-plugin/0.log"
May 28 08:41:51.391223 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:51.391128    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vx4q6_ceab5153-c91d-46fc-89ba-a795bd986c0e/node-exporter/0.log"
May 28 08:41:51.421052 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:51.421024    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vx4q6_ceab5153-c91d-46fc-89ba-a795bd986c0e/kube-rbac-proxy/0.log"
May 28 08:41:51.467516 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:51.467497    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_node-exporter-vx4q6_ceab5153-c91d-46fc-89ba-a795bd986c0e/init-textfile/0.log"
May 28 08:41:52.122482 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:52.122452    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-869899b58f-86bv9_5a6bfad0-ca17-41bb-995f-73538d18a6c5/thanos-query/0.log"
May 28 08:41:52.152040 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:52.152011    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-869899b58f-86bv9_5a6bfad0-ca17-41bb-995f-73538d18a6c5/kube-rbac-proxy-web/0.log"
May 28 08:41:52.181191 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:52.181101    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-869899b58f-86bv9_5a6bfad0-ca17-41bb-995f-73538d18a6c5/kube-rbac-proxy/0.log"
May 28 08:41:52.208461 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:52.208434    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-869899b58f-86bv9_5a6bfad0-ca17-41bb-995f-73538d18a6c5/prom-label-proxy/0.log"
May 28 08:41:52.236701 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:52.236649    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-869899b58f-86bv9_5a6bfad0-ca17-41bb-995f-73538d18a6c5/kube-rbac-proxy-rules/0.log"
May 28 08:41:52.266718 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:52.266688    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-monitoring_thanos-querier-869899b58f-86bv9_5a6bfad0-ca17-41bb-995f-73538d18a6c5/kube-rbac-proxy-metrics/0.log"
May 28 08:41:53.668759 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.668650    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:41:53.672008 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.671991    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/3.log"
May 28 08:41:53.883559 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.883528    2577 kubelet.go:2537] "SyncLoop ADD" source="api" pods=["openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"]
May 28 08:41:53.883985 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.883965    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" containerName="gather"
May 28 08:41:53.883985 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.883986    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" containerName="gather"
May 28 08:41:53.884130 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.884003    2577 cpu_manager.go:401] "RemoveStaleState: containerMap: removing container" podUID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" containerName="copy"
May 28 08:41:53.884130 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.884011    2577 state_mem.go:107] "Deleted CPUSet assignment" podUID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" containerName="copy"
May 28 08:41:53.884130 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.884087    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" containerName="gather"
May 28 08:41:53.884130 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.884102    2577 memory_manager.go:356] "RemoveStaleState removing state" podUID="818596ea-a57a-48b1-8e83-e7e67e1dcd0c" containerName="copy"
May 28 08:41:53.889129 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.889105    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:53.892679 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.892646    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6dr7c\"/\"openshift-service-ca.crt\""
May 28 08:41:53.893872 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.893852    2577 reflector.go:430] "Caches populated" type="*v1.ConfigMap" reflector="object-\"openshift-must-gather-6dr7c\"/\"kube-root-ca.crt\""
May 28 08:41:53.893981 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.893855    2577 reflector.go:430] "Caches populated" type="*v1.Secret" reflector="object-\"openshift-must-gather-6dr7c\"/\"default-dockercfg-5g6k2\""
May 28 08:41:53.901211 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:53.901188    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"]
May 28 08:41:54.047561 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.047533    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"kube-api-access-qbfcc\" (UniqueName: \"kubernetes.io/projected/cdf9af78-700f-4b9c-96b9-06084468392c-kube-api-access-qbfcc\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.047770 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.047571    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cdf9af78-700f-4b9c-96b9-06084468392c-proc\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.047770 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.047590    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cdf9af78-700f-4b9c-96b9-06084468392c-lib-modules\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.047770 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.047698    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdf9af78-700f-4b9c-96b9-06084468392c-sys\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.047770 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.047750    2577 reconciler_common.go:251] "operationExecutor.VerifyControllerAttachedVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cdf9af78-700f-4b9c-96b9-06084468392c-podres\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.148505 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.148465    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdf9af78-700f-4b9c-96b9-06084468392c-sys\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.148743 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.148534    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cdf9af78-700f-4b9c-96b9-06084468392c-podres\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.148743 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.148600    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"sys\" (UniqueName: \"kubernetes.io/host-path/cdf9af78-700f-4b9c-96b9-06084468392c-sys\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.148743 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.148605    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"kube-api-access-qbfcc\" (UniqueName: \"kubernetes.io/projected/cdf9af78-700f-4b9c-96b9-06084468392c-kube-api-access-qbfcc\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.148743 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.148678    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cdf9af78-700f-4b9c-96b9-06084468392c-proc\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.148743 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.148704    2577 reconciler_common.go:224] "operationExecutor.MountVolume started for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cdf9af78-700f-4b9c-96b9-06084468392c-lib-modules\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.148743 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.148727    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"podres\" (UniqueName: \"kubernetes.io/host-path/cdf9af78-700f-4b9c-96b9-06084468392c-podres\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.149037 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.148812    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"proc\" (UniqueName: \"kubernetes.io/host-path/cdf9af78-700f-4b9c-96b9-06084468392c-proc\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.149037 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.148839    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"lib-modules\" (UniqueName: \"kubernetes.io/host-path/cdf9af78-700f-4b9c-96b9-06084468392c-lib-modules\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.163217 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.163189    2577 operation_generator.go:615] "MountVolume.SetUp succeeded for volume \"kube-api-access-qbfcc\" (UniqueName: \"kubernetes.io/projected/cdf9af78-700f-4b9c-96b9-06084468392c-kube-api-access-qbfcc\") pod \"perf-node-gather-daemonset-qmq64\" (UID: \"cdf9af78-700f-4b9c-96b9-06084468392c\") " pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.199218 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.199183    2577 util.go:30] "No sandbox for pod can be found. Need to start a new one" pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.219557 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.219531    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console_downloads-57bd96b888-rgw82_41681891-ca0e-4f41-9550-e312ceb051a7/download-server/0.log"
May 28 08:41:54.335688 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.335633    2577 kubelet.go:2544] "SyncLoop UPDATE" source="api" pods=["openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"]
May 28 08:41:54.338100 ip-10-0-137-158 kubenswrapper[2577]: W0528 08:41:54.338067    2577 manager.go:1169] Failed to process watch event {EventType:0 Name:/kubepods.slice/kubepods-podcdf9af78_700f_4b9c_96b9_06084468392c.slice/crio-340ef053f07e9710e55a6fadfdc3d25569378535a2f2c372a5863de4dd5ebf8c WatchSource:0}: Error finding container 340ef053f07e9710e55a6fadfdc3d25569378535a2f2c372a5863de4dd5ebf8c: Status 404 returned error can't find the container with id 340ef053f07e9710e55a6fadfdc3d25569378535a2f2c372a5863de4dd5ebf8c
May 28 08:41:54.339776 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.339757    2577 provider.go:93] Refreshing cache for provider: *credentialprovider.defaultDockerConfigProvider
May 28 08:41:54.981156 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.981119    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64" event={"ID":"cdf9af78-700f-4b9c-96b9-06084468392c","Type":"ContainerStarted","Data":"be5e7315f847d334cd7ea1d8302997924966ea4978476216a13f7e3da0598d02"}
May 28 08:41:54.981156 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.981158    2577 kubelet.go:2569] "SyncLoop (PLEG): event for pod" pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64" event={"ID":"cdf9af78-700f-4b9c-96b9-06084468392c","Type":"ContainerStarted","Data":"340ef053f07e9710e55a6fadfdc3d25569378535a2f2c372a5863de4dd5ebf8c"}
May 28 08:41:54.981590 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.981239    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="not ready" pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:41:54.999421 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:54.999368    2577 pod_startup_latency_tracker.go:104] "Observed pod startup duration" pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64" podStartSLOduration=1.999342581 podStartE2EDuration="1.999342581s" podCreationTimestamp="2026-05-28 08:41:53 +0000 UTC" firstStartedPulling="0001-01-01 00:00:00 +0000 UTC" lastFinishedPulling="0001-01-01 00:00:00 +0000 UTC" observedRunningTime="2026-05-28 08:41:54.997939116 +0000 UTC m=+1793.434118282" watchObservedRunningTime="2026-05-28 08:41:54.999342581 +0000 UTC m=+1793.435521818"
May 28 08:41:55.432037 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:55.432009    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vcwq4_48d7931c-9e47-424d-aaaf-1d3c90add99a/dns/0.log"
May 28 08:41:55.458345 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:55.458320    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_dns-default-vcwq4_48d7931c-9e47-424d-aaaf-1d3c90add99a/kube-rbac-proxy/0.log"
May 28 08:41:55.538976 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:55.538953    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-dns_node-resolver-k9tpr_07b247ee-4fc0-4b9e-ae81-9f260ce894fc/dns-node-resolver/0.log"
May 28 08:41:56.054939 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:56.054908    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-image-registry_node-ca-bp4g8_a4b772f7-1b71-4e3b-b2a9-fbf33edfce66/node-ca/0.log"
May 28 08:41:56.793139 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:56.793108    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress_router-default-7df5fc48dd-7nhw9_2b55796c-941d-42d5-a985-40d51c23e675/router/0.log"
May 28 08:41:57.189457 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:57.189431    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ingress-canary_ingress-canary-sjzdf_50981871-e522-44dc-b0a5-209d7ec9958f/serve-healthcheck-canary/0.log"
May 28 08:41:57.960510 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:57.960471    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v8cmf_3fc18f79-4b6c-47d7-ad7f-65341185ec83/kube-rbac-proxy/0.log"
May 28 08:41:58.004366 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:58.004335    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v8cmf_3fc18f79-4b6c-47d7-ad7f-65341185ec83/exporter/0.log"
May 28 08:41:58.053953 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:41:58.053906    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-insights_insights-runtime-extractor-v8cmf_3fc18f79-4b6c-47d7-ad7f-65341185ec83/extractor/0.log"
May 28 08:42:00.994356 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:00.994328    2577 kubelet.go:2658] "SyncLoop (probe)" probe="readiness" status="ready" pod="openshift-must-gather-6dr7c/perf-node-gather-daemonset-qmq64"
May 28 08:42:02.223890 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:02.223864    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:42:02.230203 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:02.230175    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-console-operator_console-operator-76dff48d59-xkwsz_601ae44f-2916-47d8-9538-61dd910c49d7/console-operator/2.log"
May 28 08:42:03.085990 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:03.085959    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-qzjtr_596d1656-db0e-41c6-8dc1-9926d1b470c9/kube-storage-version-migrator-operator/1.log"
May 28 08:42:03.086871 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:03.086852    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-kube-storage-version-migrator-operator_kube-storage-version-migrator-operator-565644f49-qzjtr_596d1656-db0e-41c6-8dc1-9926d1b470c9/kube-storage-version-migrator-operator/0.log"
May 28 08:42:04.247577 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:04.247550    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8vp6h_48fe84ba-9a15-4a3d-a449-fb2694fbdfcf/kube-multus-additional-cni-plugins/0.log"
May 28 08:42:04.273498 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:04.273473    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8vp6h_48fe84ba-9a15-4a3d-a449-fb2694fbdfcf/egress-router-binary-copy/0.log"
May 28 08:42:04.298716 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:04.298686    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8vp6h_48fe84ba-9a15-4a3d-a449-fb2694fbdfcf/cni-plugins/0.log"
May 28 08:42:04.328712 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:04.328685    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8vp6h_48fe84ba-9a15-4a3d-a449-fb2694fbdfcf/bond-cni-plugin/0.log"
May 28 08:42:04.354968 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:04.354940    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8vp6h_48fe84ba-9a15-4a3d-a449-fb2694fbdfcf/routeoverride-cni/0.log"
May 28 08:42:04.381001 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:04.380980    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8vp6h_48fe84ba-9a15-4a3d-a449-fb2694fbdfcf/whereabouts-cni-bincopy/0.log"
May 28 08:42:04.407201 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:04.407174    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-additional-cni-plugins-8vp6h_48fe84ba-9a15-4a3d-a449-fb2694fbdfcf/whereabouts-cni/0.log"
May 28 08:42:04.855240 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:04.855208    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_multus-mbdnp_091802e9-2e2c-4c10-a42c-88623f528cd0/kube-multus/0.log"
May 28 08:42:05.000019 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:04.999948    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mlpsc_90e9cadb-328a-47e1-a119-bae6987ac229/network-metrics-daemon/0.log"
May 28 08:42:05.032155 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:05.032127    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-multus_network-metrics-daemon-mlpsc_90e9cadb-328a-47e1-a119-bae6987ac229/kube-rbac-proxy/0.log"
May 28 08:42:06.076362 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:06.076331    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hpqw6_21817e48-7f6c-4c5b-bd3c-ada88d6de024/ovn-controller/0.log"
May 28 08:42:06.140901 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:06.140873    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hpqw6_21817e48-7f6c-4c5b-bd3c-ada88d6de024/ovn-acl-logging/0.log"
May 28 08:42:06.162120 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:06.162083    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hpqw6_21817e48-7f6c-4c5b-bd3c-ada88d6de024/kube-rbac-proxy-node/0.log"
May 28 08:42:06.186770 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:06.186727    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hpqw6_21817e48-7f6c-4c5b-bd3c-ada88d6de024/kube-rbac-proxy-ovn-metrics/0.log"
May 28 08:42:06.208120 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:06.208093    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hpqw6_21817e48-7f6c-4c5b-bd3c-ada88d6de024/northd/0.log"
May 28 08:42:06.233070 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:06.233046    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hpqw6_21817e48-7f6c-4c5b-bd3c-ada88d6de024/nbdb/0.log"
May 28 08:42:06.255381 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:06.255335    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hpqw6_21817e48-7f6c-4c5b-bd3c-ada88d6de024/sbdb/0.log"
May 28 08:42:06.357667 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:06.357571    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-ovn-kubernetes_ovnkube-node-hpqw6_21817e48-7f6c-4c5b-bd3c-ada88d6de024/ovnkube-controller/0.log"
May 28 08:42:07.877989 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:07.877962    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-diagnostics_network-check-target-nkgkq_ecd64268-adf3-40c0-aaaf-510d4c689d8c/network-check-target-container/0.log"
May 28 08:42:08.863828 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:08.863798    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-network-operator_iptables-alerter-p6k96_dca0512e-79d3-4c51-9f05-cc6c64fe4f85/iptables-alerter/0.log"
May 28 08:42:09.657376 ip-10-0-137-158 kubenswrapper[2577]: I0528 08:42:09.657348    2577 log.go:25] "Finished parsing log file" path="/var/log/pods/openshift-cluster-node-tuning-operator_tuned-fhw4w_18155ab5-6646-4bf6-88af-c74297a72a1e/tuned/0.log"